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ABSTRACT 


This  research  investigated  how  different  experts  in  a  single  domain  chose  their 
individual  subjective  evaluation  criteria  of  a  highly  aggregate  task  based  upon  their 
individual  differences.  The  Conning  Officer  Virtual  Environment  (COVE)  was  utilized 
to  provide  a  domain  of  experts  and  a  subjectively  evaluated  task.  116  expert  ship- 
handlers  were  investigated  to  understand  how  their  personality  affects  their  evaluation  of 
a  novice  performing  an  underway  replenishment  (UNREP).  The  experts  were  issued  a 
survey  that  inventoried  their  personality,  UNREP  evaluation  criteria,  and  ship  handling 
style.  In  general,  the  participant  experts  were  lower  in  Neuroticism  and  higher  in 
Extraversion  and  Conscientiousness  than  the  average  adult.  Extraversion  appeared  to  be 
correlated  with  the  expert's  desire  to  use  Sensory  Input  as  a  critical  evaluation  criterion 
(p  =  .18)  while  Openness  was  correlated  with  Analytical  Input  (p  =  .16)  and  UNREP  style 
(p  =  .16)  as  critical  evaluation  factors.  Also  correlated  with  UNREP  style  was 
Agreeableness  (p  =  .16).  Finally,  the  expert's  level  of  Conscientiousness  correlated  with 
the  critical  evaluation  criteria  of  Analytical  Input  (p  =  .17)  and  Sensory  Input  (p  =  .39). 
Results  from  this  research  provide  insight  to  the  link  between  observed  behavior  and  its 
subjective  evaluation  and  will  allow  COVE's  programmers  to  develop  an  Intelligent 
Tutoring     System    (ITS)    that    will     customize    the    automated    training    process. 
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I.  INTRODUCTION 


A.         MOTIVATION 

The  military  services  have  historically  been  an  apprenticeship  system.  Beginning 
with  the  early  days  of  sail,  United  States  Naval  Midshipmen  would  spend  several  years 
serving  aboard  a  single  ship  with  a  single  captain.  Under  the  tutelage  and  guidance  of  his 
master  captain,  the  apprentice  midshipman  would  learn  the  art  of  sail  and  war.  A 
midshipman  would  be  promoted  to  the  rank  of  an  officer  of  the  line  only  after  gaining  his 
captain's  full  trust  and  confidence  in  his  knowledge  and  abilities.  This  type  of  highly 
specialized  training  required  vast  resources  and  the  dedication  of  numerous  personnel. 

Whether  a  sailor,  marine,  soldier,  or  airman,  the  United  States  Military  warrior  of 
the  new  millennium  is  required  to  do  more  with  less.  Fewer  troops,  weapons,  training 
time,  and  fiscal  resources  are  requiring  the  armed  forces  to  re-evaluate  every  facet  of 
operations.  In  particular,  budget  draw  downs  and  the  increasing  complexity  of  hardware 
necessitates  the  need  to  create  cost-effective  training  alternatives.  As  computing  power 
and  speed  increase,  the  desire  for  utilizing  computers  as  a  beneficial  training  tool  also 
increases.  Using  modern  computers'  ever  increasing  high  fidelity  virtual  environments 
(VEs)  as  a  training  tool  for  performing  spatial  and  cognitive  tasks  are  a  particular  area  of 
increasing  demand  since  VEs  provide  the  potential  capability  for  a  trainee  to  practice  and 
master  complex  and  highly  dangerous  tasks  safely,  efficiently,  and  economically 
[CAIR96]. 
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B.         OBJECTIVE 

"The  best  known  generalization  in  human  learning  is  that  practice  makes  perfect" 
[ANNE89].  The  caveat  to  this  cliche  is  that  the  student  is  practicing  the  right  task  in  the 
right  ways.  The  ability  to  properly  react  to  any  situation  requires  expert  guidance  and 
proper  intervention  at  critical  points  during  training.  Without  a  good  teacher,  practice 
alone  is  not  always  enough  to  become  competent  at  a  complex  task.  Even  more 
devastating  is  the  possibility  that  the  student  will  get  worse  at  the  learned  task  and 
experience  a  negative  training  transfer  [BOLD87]. 

While  VE  is  a  relatively  new  training  tool,  a  VE  training  system  is  not 
pragmatically  different  from  any  previous  generation  of  training  tool.  VE  training  must, 
just  like  any  other  training  system,  provide  students  with  the  skills  and  knowledge 
required  to  meet  the  demands  of  the  trained  task  and  the  needs  of  the  overriding 
organization  [CAIR96].  While  any  form  of  training  requires  several  key  components  to 
be  effective,  one  of  the  most  essential  steps  to  developing  a  successful  training  program  is 
providing  quality  feedback  via  instruction  and  evaluation. 

The  Conning  Officer  Virtual  Environment  (COVE)  Ship  Handling  Trainer  is  one 
example  of  a  VE  that  provides  an  economically  attractive  alternative  to  traditional 
methods  of  practicing  ship  handling  while  providing  an  integrated  means  of  instruction. 
COVE,  currently  under  development  by  Naval  Air  Warfare  Training  Systems  Division 
(NAWC-TSD),  simulates  ship-handling  scenarios  where  the  trainee  is  immersed  in  a  VE, 
complete  with  an  integrated  intelligent  tutoring  system  (ITS),  in  the  form  of  a  simulated 
interactive  commanding  officer.  COVE  is  a  flexible  and  portable  unit  that  is  intended  to 
build   and   reinforce    ship-handling   skills   with   minimal   requirements    for   instructor 


intensiveness  and  costly  ship  resources  [MEAD99].  If  the  deployed  implementation  of 
COVE  is  successful.  Junior  Officers  (JOs)  will  have  an  opportunity  to  develop  basic 
skills  and  practice  difficult  scenarios  in  a  controlled  environment  without  the  need  to 
have  entire  ships  at  sea,  saving  time,  dollars,  and  possibly  even  lives. 

"Providing  the  trainee  with  knowledge  of  the  results  is  one  of  the  most  common 
training  program  interventions  and  one  which  is  generally  believed  to  have  a  powerful 
effect  on  learning"  [ANNE89].  In  the  case  of  COVE,  quality  feedback  to  the  trainee 
requires  the  ITS  to  be  more  than  a  scripted  set  of  rules.  The  ITS  must  provide  immediate 
guidance  and  feedback  that  is  accurate  and  meets  accepted  standards,  just  as  a 
Commanding  Officer  (CO)  would  at  sea,  else  risk  the  loss  of  valuable  training  time  and  a 
possible  negative  transfer  training  experience  [TENN99]. 

ITS  feedback  should  both  conform  to  accepted,  safe  practices  and  the 
requirements  of  the  JO's  CO.  However,  the  dynamics  of  handling  a  ship  at  sea  combined 
with  individual  differences  of  COs  makes  it  difficult  to  have  a  single  standardized  set  of 
feedback  responses.  Just  as  the  original  shipmasters  trained  their  apprentices  uniquely, 
today's  COs  train  their  JOs  according  to  their  predilections.  Different  COs  will  have 
different  benchmarks  based  upon  their  own  style  of  expertise,  experiences,  and 
personality,  resulting  in  different  COs  evaluating  the  same  evolution  differently  [NPS99]. 
In  order  to  gain  maximum  benefit  for  the  fleet,  COVE's  ITS  must  be  flexible  enough  to 
meet  the  needs  of  the  different  fleet  experts. 

While  a  prime  example,  COVE  is  just  a  single  example  of  a  trainer  that  requires 
extensive  knowledge  and  that  has  infinitely  many  ways  to  arrive  at  a  "correct"  solution 
that  is  "correct"  only  in  the  eyes  of  the  evaluator.  Topics  easily  range  from  driving  ships, 


to  land  navigation,  to  philosophy.  Essentially,  anything  that  involves  asynchronous 
student-paced  instruction  and  training  of  a  highly  complex  aggregate  task  that  involves 
subjectivity  in  evaluation  can  benefit  from  the  relationships  explored  by  the  COVE  ITS 
and  student. 

This  study  investigates  how  simulator  performance  evaluation  should  be  modeled 
based  upon  the  personality  composition  of  the  evaluating  expert.  With  respect  to  the 
COVE  trainer,  it  is  desired  to  understand  the  different  evaluation  criteria  used  by  different 
COs  and  its  relationship  to  their  personality,  ultimately  resulting  in  a  more  accurate  ITS 
where  the  Virtual  CO  (VCO)  could  approximate  a  wide  range  of  real  world  COs. 


To  be  true  to  form,  one  would  have  to  have  the  many  different  styles  of 
COs  within  the  system  and  the  ability  to  choose  which  one  you  need.  At 
the  one  end  would  be  the  screamer  that  we  may  be  most  familiar  with  who 
will  throw  you  off  the  bridge  if  you  go  too  far,  and  at  the  other  end  would 
be  the  true  mentor  who  lets  you  get  to  the  point  of  no  return  only  to  help 
you  avoid  the  collision  that  you  thought  was  inevitable. 

Commanding  Officer  of  an  LPD 


This  accurate  modeling  results  in  more  effective  trainer  time  by  teaching  the  JO  the  same 
lessons  his  real  world  CO  would  teach,  increasing  the  effectiveness  and  overall  benefit  of 
the  trainer. 


C.         THESIS  QUESTIONS 

The  following  questions  are  addressed  in  this  thesis: 


•  Is   there   a   relationship   between   one's   personality   and    one's 
expertise? 

•  If  such  a  relationship  exists,  can  it  be  quantified? 

•  Does    it    extend    beyond    individual    expertise    to    the    expert's 
evaluation  of  others'  performance? 

•  What  is  the  range  of  characteristics  of  different  ship  driving  styles? 


Addressing  these  questions  is  the  first  step  in  building  a  more  accurate  ITS  for 
COVE.  Since  this  research  is  only  the  initial  exploration  between  human  behavior  and 
expert  evaluation,  it  is  intended  to  begin  the  initial  compilation  of  a  database  for  the 
COVE  ITS.  Understanding  the  answers  to  the  aforementioned  questions  will  provide 
COVE's  ITS  programmers  with  a  realistic  model  to  base  various  prototypical  VCOs 
upon.  Furthermore,  these  answers  also  lay  the  foundation  for  automating  the  relationship 
and  increasing  the  fidelity  between  instructor  and  student  in  any  VE  with  an  ITS.  This 
added  insight  will  help  mate  the  ITS  with  the  student,  potentially  increasing  positive 
training  transfer  for  any  VE  training  system. 


D.         APPROACH 

In  order  to  answer  the  questions  posed  by  this  research,  knowledge  about  the 
relationship  between  experts  and  novices  is  required.  Along  with  the  nature  of  expertise, 


knowledge  about  personality  and  its  measurement  must  be  obtained.  Furthermore,  the 
scope  of  this  work  requires  an  understanding  of  the  sociological  domain  within  which  the 
experts  and  novices  reside. 

Naval  officers  achieve  the  prestige  of  command  only  by  displaying  sustained 
superior  performance,  primarily  at  sea.  As  the  senior  ship  driver  aboard,  and  the  one 
person  ultimately  responsible  for  any  mishap,  the  Commanding  Officer  (CO)  is  the 
resident  ship-handling  expert.  How  a  ship  is  driven  by  any  of  the  ship's  officers  is  a 
direct  statement  about  the  ship  handling  abilities  of  its  CO. 

Few  evolutions  make  or  break  a  CO's  reputation  like  the  UNREP  approach  to  the 
auxiliary  replenishment  ship  since  the  approach  is  a  calling  card  for  the  CO's  style  and 
ability.  While  all  UNREP  experts  achieve  the  same  ultimate  end  goal  of  coming  along 
side  the  replenishment  ship,  different  COs  accomplish  this  task  differently.  Some  prefer 
to  "John  Wayne"  with  large  speed  differentials  and  small  distances  from  "rubbing  paint" 
while  others  prefer  more  of  a  slow  and  steady  approach.  Some  COs  base  decisions  on 
aggregate  big  picture  data  while  others  require  more  specific  input. 

While  the  UNREP  is  one  of  the  greatest  showcases  of  skills  for  the  surface 
warfare  officer,  it  is  also  one  of  the  most  dangerous  where  the  potential  for  loss  of  life 
and  damage  to  not  only  one  but  also  two  ships  is  extremely  high.  The  ability  to  actually 
practice  this  formidable  task  at  sea  is  limited  and  can  quickly  evolve  into  a  situation  too 
complex  for  a  junior  officer  to  handle.  These  criteria  result  in  good  VE  training  being 
crucial  and  indicate  that  UNREP  is  an  excellent  VE  candidate  since  it  allows  the 
opportunity  for  officers  to  develop  prerequisite  skills  in  a  safe  and  controlled  environment 
with  minimal  operating  cost. 


Because  of  all  of  the  aforementioned  factors,  UNREP  was  the  vehicle  for  this 
research  and  analysis.  Its  importance  in  the  sociological  domain  of  the  ship  driver  also 
makes  it  suitable  to  determine  the  correlations  with  ship  driver  personality.  An  analysis 
of  the  expert  evaluation  of  an  UNREP  approach  as  performed  by  a  less-experienced  JO 
was  correlated  with  the  personality  of  the  expert  to  answer  the  thesis  questions.  Results 
can  be  directly  applied  to  COVE's  existing  ITS  during  COVE  simulated  UNREPs. 


SUMMARY  OF  CHAPTERS 


The  remainder  of  this  thesis  is  broken  down  into  the  following  chapters: 


Chapter  II  provides  background  information  on  the  mental  and 
behavioral  processes  invoked  during  UNREP  and  other  complex 
tasks.  First,  a  review  of  the  basic  components  of  an  UNREP  is 
performed  followed  by  a  summary  of  COVE  and  ITS  previous 
research.  Next,  a  more  in-depth  view  of  the  differences  between 
experts  and  novices  is  explored  in  order  to  understand  the 
differences  between  COs  and  JOs.  Additionally,  individual 
differences  and  their  effects  on  decision  making  are  explored. 
Finally,  personality  and  its  measurement  are  discussed  in  order  to 
understand  how  individual  expert  COs  are  different  from  each 
other. 


• 


Chapter  III  discusses  the  apparatus  utilized  to  gather  information 
for  this  research.  Reasons  for  selection,  design,  and  development 
are  covered  for  the  two  data  collection  tools,  the  NEO-FFI 
personality  inventory  and  the  Ship  Handler  Evaluation  Survey. 

Chapter  IV  delineates  the  methods  utilized  for  data  collection  and 
analysis.  An  explanation  detailing  the  administration  of  the  survey 
is  provided  in  addition  to  a  summary  of  the  construction  of  the  data 
package. 


• 


Chapter  V  summarizes  the  results  from  the  data  collection  and 
analysis.  Results  are  provided  detailing  the  personality 
characteristics  of  the  participant  expert  ship  handlers,  the  critical 
evaluation  criteria  utilized  by  expert  ship-handlers  for  evaluation  of 
novice  JOs,  and  significant  correlations  observed  between 
personality  and  critical  evaluation  criteria. 

Chapter  VI  presents  a  final  discussion  of  the  results  of  this  thesis 
and  describes  areas  requiring  further  research.  Answers  to  thesis 
questions  proposed  by  this  research  are  addressed  in  addition  to  the 
possible  ramifications  of  this  research. 


II.  BACKGROUND 


A.    THE  UNREP  EVOLUTION 

The  UNREP  evolution,  while  complex  and  dangerous  during  execution,  is  a 
particularly  straightforward  task.  Two  ships,  an  approach  vessel  and  a  replenishment 
vessel  execute  the  evolution.  The  approach  vessel  is  a  warship  that  requires 
replenishment  of  its  fuel  and  or  stores.  The  replenishment  vessel  is  usually  a  refueling 
tanker.  The  overall  goal  of  the  evolution  is  for  the  approaching  vessel  to  come  within 
close  proximity  of  the  replenishing  vessel  and  bring  on  fuel  and  other  supplies  with  out 
any  damage  to  personnel  or  equipment. 

The  UNREP  is  composed  of  distinct  phases  consisting  of  preparations,  waiting, 
approach,  alongside,  and  breakaway.  Figure  1  is  a  diagram  depicting  the  different  phases 
involved  in  a  plausible  UNREP  scenario  and  highlights  some  of  the  distances  involved 
between  the  two  ships  participating  in  an  UNREP.  The  evolution  actually  starts  hours 
before  the  actual  transfer  of  supplies  is  executed  by  performing  the  preparation  phase. 
Checks  of  ships  systems  and  a  pre-execution  brief  are  performed  on  both  ships  to  ensure 
that  both  the  ships  and  crews  are  prepared  to  perform  the  actual  task. 

The  next  phase,  the  waiting  phase,  is  just  prior  to  the  commencement  of  the 
approach.  During  this  phase,  the  approach  vessel  maneuvers  to  a  waiting  station  where 
the  approach  vessel  will  perform  its  last  checks  and  wait  for  a  signal  from  the 


replenishing  vessel  to  commence  the  approach.    The  waiting  station  is  usually  an  area 
approximately  1000  yards  astern  of  the  replenishment  vessel. 
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Figure  1 :  A  Diagram  of  a  possible  UNREP  evolution. 
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Once  both  ships  are  on  an  agreed  upon  course  and  speed  and  are  ready,  signals  are 
made  and  the  approach  phase  commences.  During  the  approach  phase,  the  approaching 
vessel  maneuvers  from  waiting  station  to  a  position  directly  alongside  the  replenishment 
vessel.  It  is  during  the  approach  phase  that  the  first  interaction  of  physical  forces  occurs 
between  the  two  vessels. 

Once  the  approach  is  made,  lines  connecting  the  two  ships  are  secured  and  the 
approach  phase  transitions  into  the  alongside  phase.  During  the  alongside  phase,  the 
approach  vessel  maintains  a  constant  position  relative  to  the  replenishment  vessel  during 
the  transfer  of  fuel  and  stores.  Radio  communication  is  maintained  between  both  vessels 
during  the  alongside  phase  until  transfer  is  complete  between  the  replenishment  vessel 
and  the  approach  vessel.  Transfer  time  primarily  depends  upon  the  amount  of  supplies  to 
be  transferred,  but  typically  is  less  than  an  hour. 

Once  all  supplies  are  transferred,  all  connecting  lines  between  the  two  ships  are 
cast  off,  marking  the  beginning  of  the  breakaway  phase.  During  this  final  phase,  the 
approach  vessel  maneuvers  away  from  the  replenishment  vessel.  Once  clear  of  the 
replenishment  vessel,  the  approach  vessel  is  no  longer  restricted  in  its  ability  to  maneuver 
and  can  proceed  on  its  own  independent  course  and  speed. 


B.         THE      CONNING      OFFICER      VIRTUAL      ENVIRONMENT      AND 
INTELLIGENT  TUTORING  SYSTEM 

The  COVE  trainer  is  a  direct  evolvement  of  a  previous  project  by  NAWC-TSD, 
the  Virtual  Environment  Submarine  (VESUB)  Simulator.     VESUB  was  intended  to 
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provide  a  means  for  submarine  officers  to  practice  surfaced  submarine  evolutions,  in 
particular  transit  in  and  out  of  port,  with  out  the  need  of  a  surfaced  submarine.  COVE 
combines  some  of  the  original  VESUB  visual  simulation  architecture  with  voice 
recognition  and  an  integrated  intelligent  tutoring  system.  Ideally,  COVE  is  a  portable, 
low  cost  training  solution  that  provides  the  user  with  a  high  fidelity  synthetic  ship  driving 
experience  and  requires  no  operator  monitoring  or  intervention  [MEAD99]. 

Most  previously  implemented  expert  systems  possess  limited  capability  for 
diagnosis  and  feedback  making  them  relatively  unsuitable  for  training  purposes 
[TENN99].  In  order  for  an  artificially  intelligent  (AI)  training  system  to  be  successful,  it 
must  possess  the  capability  to  learn  from  experience  by  making  human-like  associations 
requiring  a  sense  of  appropriate  output  and  understanding  of  needs,  desires,  and  emotions 
[DREY96].  A  possible  architecture  that  meets  these  criteria  incorporates  adaptive 
technology  into  a  pedagogical  agent.  An  example  of  a  first  generation  ITS  is  STEVE 
(Soar  Training  for  Virtual  Environments),  which  is  currently  under  development  by  the 
Air  Force  Research  Laboratory  [TENN99].  STEVE  is  designed  to  be  a  modular  agent 
implementation  for  the  purpose  of  instruction  in  a  variety  of  computer-based  learning 
environments  [JOHN98]. 

The  COVE  trainer  incorporates  an  ITS  in  the  form  of  a  Virtual  Commanding 
Officer  (VCO).  The  VCO  is  a  pedagogical  agent  that  instructs  the  JO  on  how  to  properly 
drive  the  ship  during  a  ship  handling  evolution  such  as  UNREP  [TENN99].  Previous 
research  investigated  three  possible  profiles  for  the  VCO  consisting  of  a  passive  VCO,  a 
proactive  VCO,  and  an  aggressive  VCO.  The  classification  of  passive,  proactive,  or 
aggressive  was  based  primarily  on  a  CO's  predilection  to  recommend  course  and  speed 
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changes  while  a  JO  was  conning  the  ship  during  an  UNREP  evolution.  The  three 
classifications  were  chosen  for  their  broad  coverage  of  different  ship  driving  styles  and 
were  intended  to  represent  the  majority  of  ship  drivers  in  the  fleet. 


C.         DIFFERENCES  BETWEEN  EXPERTS  AND  NOVICES 


I  want  the  JO  to  learn  to  drive  the  ship  the  way  I  drive  the  ship." 


Member  of  SURFLANT  staff  addressim 
NAWC-TSD  about  COVE  and  ITS. 


While  a  CO  will  usually  find  other  styles  acceptable,  he  will  prefer  his  JOs  to 
drive  the  ship  in  a  manner  similar  to  his.  Much  like  a  father  teaching  his  teenager  how  to 
drive  a  car,  the  expert  CO  will  first  instruct,  and  later  expect  the  novice  JO  to  analyze 
data  and  make  decisions  in  the  same  fashion  as  the  CO.  These  expectations  are  the  basis 
for  the  expert  CO's  evaluation  of  the  novice  JO  and  are  shaped  by  the  CO's  expertise. 

UNREPs  are  dynamic,  complex  tasks  and  UNREP  skill  cannot  be  neatly 
categorized  under  a  single  type  of  expertise.  Expertise  itself  is  diverse  and  is  segregated 
into  the  four  following  categories:  [CHI88] 


•  PRACTICAL  EXPERTISE  that  primarily  deals  with  motor  skills 
or  mental  skills.  Examples  of  practical  skills  are  typing, 
memorizing  restaurant  orders,  or  mental  calculation.  This  type  of 
expertise  often  allows  for  parallel  thought  processing. 

•  PROBLEM  SOLVING  EXPERTISE  requiring  specific  domain 
related  knowledge.  Examples  of  problem  solving  expertise  are 
computer  programming  or  solving  physics  problems. 
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•  ILL-DEFINED  EXPERTISE  that  requires  decisions  under 
uncertainty,  such  as  when  some  uncontrolled  intervening  event 
occurs  between  the  choice  and  the  outcome.  An  example  of  an  ill- 
defined  problem  is  predicting  stock  market  performance  [CHI88]. 

•  DIAGNOSTIC  EXPERTISE  where  metacognition  is  required  to 
accurately  access  the  reason  for  a  given  circumstance  or  set  of 
facts.  An  example  of  diagnostic  expertise  is  properly  accessing  an 
illness  or  medical  condition  from  x-rays  or  symptoms. 


All  four  of  the  previous  expertise  categories  apply  to  an  UNREP  evolution. 
Commands  are  issued  and  executed  with  practical  expertise.  Estimating  times  and  speeds 
in  open-ocean  utilize  problem-solving  techniques.  Given  the  dynamic  nature  of  an 
UNREP  due  to  the  uncontrollable  forces  of  nature  and  the  interactions  of  two  separate 
independent  ship  drivers  simultaneously,  UNREPs  require  both  ill-defined  and  diagnostic 
expertise.  A  novice  JO  must  demonstrate  proficiency  of  all  types  of  expertise  in  order  to 
receive  a  favorable  evaluation  of  the  UNREP  evolution  from  his  CO. 

Expert  ship  handlers  usually  distinguish  themselves  from  novices  by  determining 
the  quickest,  most  efficient  courses  of  action,  and  when  a  ship  handling  evolution  is 
getting  out  of  control.  In  distinguishing  themselves  from  novices,  experts,  regardless  of 
the  area  of  expertise,  share  common  traits.  These  commonalties  are  summarized  as: 

•  Experts  excel  mainly  at  their  own  domains. 

•  Experts  perceive  large  meaningful  patterns  in  their  domains. 

•  Experts  quickly  solve  problems  with  little  error. 

•  Experts  have  superior  short-term  and  long-term  memory. 

•  Experts  see  and  represent  a  problem  at  a  deep  (more  principled) 
level. 
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•  Experts    spend    a    great    deal    of    time    analyzing    a    problem 
qualitatively. 

•  Experts  have  strong  self-monitoring  skills. 


These  traits  usually  result  in  an  expert  performing  a  task  quicker  and  with  fewer  errors 
[CHI88]. 

Even  though  experts  distinguish  themselves  apart  from  novices  in  common  ways, 
there  are  still  wide  variabilities  amongst  the  experts  themselves.  These  variabilities  are 
unique  to  each  expert  and  are  often  referred  to  as  individual  differences.  These 
differences  influence  how  the  expert  responds  to  situations,  teaches  his  novices,  and 
evaluates  his  trainees'  proficiency.  Understanding  individual  differences  of  the  CO  are 
critical  inputs  to  producing  useful  feedback  for  the  JO. 


D.         DECISION-MAKING  AND  INDIVIDUAL  DIFFERENCES 

A  study  performed  by  the  U.S.  Army  Research  Institute  for  the  Behavioral  and 
Social  Sciences  assessing  how  senior  Army  officers  made  critical  battlefield  decisions 
discovered  that  not  all  experts  analyze  situations  and  make  decisions  the  same  way 
[COHE96].  Most  experts  generally  fall  into  two  completely  different  paradigms.  Some 
experts  follow  an  analytical  approach  where  decision-making  is  characterized  by 
attempting  to  use  rational  and  computational  methods.  In  contrast,  a  recognition  expert 
would  attempt  to  make  decisions  based  on  fitting  the  situation  into  a  known  pattern  and 
responding  with  a  familiar  label  or  plan  of  action. 
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Another  U.S.  Army  Research  Institute  for  the  Behavioral  and  Social  Sciences 
study  investigated  the  effects  of  expertise,  cognitive  style,  and  mission  on  what 
information  is  used  by  senior  Army  officers  during  tactical  decision  making  in  an  attempt 
to  develop  a  tactical  decision  aid  [MICH88].  Their  research  indicated  that  a  tactical 
decision  aid  must  be  adaptable  to  individual  differences  such  as  personality,  cognitive 
style,  and  preferences  for  sensory  modality  and  communication  mode.  These  findings 
correlate  with  the  research  on  how  Army  officers  performed  under  stressful  situations. 
Their  findings  showed  that  personality  exhibited  some  consistent  patterns  of  response  to 
stressful  situations.  Their  research  assumed  that  there  is  a  reciprocal  causality  between 
individual,  situational,  and  response  variables  and  that  the  way  an  individual  responds  to 
a  situation  is  directly  affected  by  the  individual's  personality. 


E.         THE  FIVE  FACTOR  MODEL  OF  PERSONALITY  AND  FEEDBACK 

The  UNREP  is  an  extremely  stressful  evolution  for  both  novice  and  expert.  Since 
an  expert  CO  will  perform  an  UNREP  based  upon  his  expertise  paradigm,  which  is 
shaped  by  his  individual  characteristics,  it  is  necessary  to  study  the  COs  personality. 
Personality  is  often  explained  by  the  Five-Factor  Model  (FFM),  which  describes 
personality  in  terms  of  five  distinct  personality  traits.  The  "  FFM  originated  in  initial 
works  by  Fiske  (1949),  Norman  (1963),  and  Tuppes  and  Christal  (1963),  who  produced  a 
highly  stable  structure  with  five  factors"  [SALG97].  The  FFM  is  extremely  attractive  due 
to  its  empirical  roots.  While  most  models  are  derived  from  theoretical  perspectives,  the 
lexical  FFM  has  a  theoretically  neutral  position  [WTDI97]. 
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The  Revised  NEO  Personality  Inventory  (NEO  PI-R)  is  modeled  after  the  FFM.  It 
is  a  widely  accepted  measure  of  personality  developed  by  Dr.  Paul  Costa  and  Dr.  Robert 
McCrae,  assesses  personality  in  terms  of  Neuroticism,  Extraversion,  Openness, 
Agreeableness,  and  Conscientiousness.  The  five  personality  factors  are  described  in  the 
following  way: 


•  EXTRAVERSION  is  the  factor  that  describes  people  who  are  rated 
by  their  peers  as  "sociable,  fun-loving,  affectionate,  friendly,  and 
talkative"  [MCCR87]  versus  "reserved,  timid,  and  quiet" 
[SALG97]. 

•  People  high  in  AGREEABLENESS  are  forgiving,  lenient, 
sympathetic,  agreeable,  and  softhearted,  according  to  peer  ratings 
[MCCR87].  Peers  describe  those  low  in  Agreeableness  in  more 
negative  terms:  ruthless,  uncooperative,  suspicious,  and  stingy. 

•  Peers  describe  people  high  in  CONSCIENTIOUSNESS  as  careful, 
well  organized,  punctual,  ambitious,  and  persevering  [MCCR87]. 
Conscientiousness  "includes  both  proactive  (hardworking, 
ambitious)  and  inhibitive  (dutiful,  scrupulous)  aspects" 
[MCCR89J. 


• 


People  who  score  high  on  NEUROTICISM  typically  report 
negative  emotions  such  as  worry,  insecurity,  self-consciousness, 
and  tempermentalness  [MCCR87]  whereas  people  with  low 
Neuroticism  are  calm,  self-confident,  and  cool  [SALG97]. 

•  The  final  factor  in  this  model  is  OPENNESS.  Adjectives  from 
lexical  studies  that  describe  this  factor  include  "original, 
imaginative,  broad  interests,  and  daring"  [MCCR87].  "Openness 
defines  individuals  who  are  creative  curious,  and  cultured  versus 
practical  with  narrow  interests.  [SALG97]" 


The  five  factors  of  personality  have  implications  for  occupational  performance 
and  therapy.  Most  research  studying  the  relationship  between  personality  and  job 
performance  only  attempts  to  correlate  quality  of  job  performance  with  personality 
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[CLON96].  Two  meta-analytic  studies  by  Barrick  and  Mount  (1991)  and  Tett  and 
Jackson  (1991)  find  that  Conscientiousness  is  the  only  predictor  of  quality  of  job 
performance  [RUST99]. 

There  is  less  research  on  the  link  between  individual  differences  and  method  of 
task  completion.  A  previous  study  showed  correlation  between  individual  differences 
and  variability  in  expertise  [NPS99].  Specifically,  ship  handler's  methods  for  performing 
an  UNREP  varied  into  two  distinct  categories,  analogical  or  analytical.  Whether  a  CO 
performed  an  UNREP  in  an  analytical  fashion,  or  an  analogical  fashion  could  be 
correlated  with  the  CO's  personality  and  cognitive  style.  A  study  investigating  the  theory 
that  personality  is  more  differentiated  at  higher  levels  of  ability  discovered  that  some 
personality  traits  are  statistically  more  variable  for  individuals  at  high  versus  low  levels  of 
ability  [AUST97].  This  research  also  showed  relationships  between  types  of  judgment 
and  FFM  factors  and  Intelligence  Quotient  (IQ). 

Rust,  1999,  investigated  the  ability  of  the  FFM  to  predict  supervisor's  ratings  of 
performance.  In  his  research,  Rust  administered  the  Orpheus;  broad-spectrum  work 
based  personality  questionnaire  to  employees.  His  findings  showed  a  correlation  between 
the  FFM  results  of  the  self-evaluation  Orpheus  and  appropriate  supervisor  ratings.  In 
evaluating  the  FFM  within  the  context  of  work  based  behavior: 


High  extroversion  people  are  generally  happier  working  with 
others  while  low  extroverts  tend  to  prefer  work  requiring 
independence. 

High  agreeableness  results  in  individuals  with  a  desire  for  a  more 
cooperative,  problem-solving  approach  the  lower  Agreeableness 
results  in  an  ability  to  make  tough  decisions. 
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People  displaying  high  openness  to  experience  seek  alternative 
solutions  and  desire  different  methods,  while  low  openness  to 
experience  individuals  desire  traditional  approaches  and  respect 
established  values. 

Low  neurotics  tend  to  disregard  feelings  of  others,  perform  better 
under  stress,  but  tend  to  lack  caution. 

High  conscientious  people  tend  to  excel  at  detailed  tasks,  but  may 
become  over  involved  in  minutiae  while  low  conscientiousness 
people  have  little  patience  for  mundane  tasks,  and  prefer  to  see  the 
big  picture. 

[RUST99] 


This  research  is  important  because  personality  is  a  factor  for  how  a  CO  learns,  and 
subsequently  trains.  The  expert  is  more  inclined  to  use  teaching  techniques  in  a  manner 
that  he  understands  the  best.  Previous  research  has  attempted  to  correlate  personality 
traits  with  various  learning  styles.  This  research  indicated  that  Extroversion  and 
Agreeableness  are  linked  with  more  active  types  of  learning  [FURN96].  Therefore,  it  is  a 
conclusion  that  Extroversion  and  Agreeableness  could  explain  active  forms  of  teaching. 
In  the  case  of  an  UNREP,  these  personality  traits  could  explain  why  some  COs  are  more 
actively  involved  with  the  JO  during  the  evolution  than  others  are. 

Salgado  analyzed  three  prior  meta-analysis  studying  the  relationship  between 
personality  and  job  criteria.  In  general,  Salgado  discovered  that  "Extraversion  is  a  valid 
predictor  of  training  proficiency  (p  =  .26),  as  are  Neuroticism  (p  =  .07),  Agreeableness 
(p  =  .10),  and  Openness  to  Experience  (p  =  .25)"  [SALG97].  Furthermore,  personality 
compatibility  between  teacher  and  student  will  potentially  affect  the  teacher's  evaluation 
of  an  evolution.  Research  demonstrates  that  students  achieved  higher  levels  in  classes 
when  teacher-student  personality  compatibility  is  high  [FURN96].     This  teacher  and 
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student  dynamic  should  also  apply  to  VE  training,  and  if  properly  modeled,  will  further 
increase  the  accuracy  of  the  training  feedback. 


F.  PERSONALITY  MEASUREMENT 

The  assessment  of  personality  is  a  major  application  of  psychology  to  real  world 
concerns  and  is  extremely  varied  in  its  administration  and  utilization.  Clinical 
psychologists  evaluate  a  patient's  personality  in  an  attempt  to  determine  if  the  patient 
possesses  abnormal  symptoms  or  feelings.  A  school  psychologist  will  assess  a  child's 
personality  in  order  to  determine  any  causes  of  possible  learning  or  adjustment  problems. 
Counseling  psychologists  attempt  to  determine  the  best  job  for  a  particular  person  by 
matching  the  individual's  needs  and  interests  with  the  requirements  of  the  position. 
Finally,  research  psychologists  assess  the  personalities  of  experiment  participants  to 
account  for  experimental  behavior  or  correlate  personality  characteristics  with  other 
measures  [SCHU90]. 

Regardless  of  the  end  goal,  some  assessment  techniques  are  more  objective  while 
some  techniques  are  wholly  subjective  and  prone  to  bias.  The  best  techniques  possess 
standardization,  reliability,  and  validity.  Standardization  insures  consistency  and 
uniformity  of  the  procedures  utilized  for  the  test  administration.  Reliability  insures 
consistency  of  results  to  the  assessment  device.  Finally,  validity  insures  that  the  test 
device  results  are  an  accurate  measurement  of  the  intended  measured  variables 
[SCHU90]. 
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Multiple  methods  exist  to  assess  personality.  A  primary  method  is  referred  to  as 
the  self-report  inventory  method.  In  the  self-report  method,  people  report  on  themselves 
by  answering  questions  about  their  feelings  and  behavior  in  a  variety  of  simulated 
situations.  The  person  taking  the  test  must  indicate  how  closely  each  item  describes  their 
own  characteristics  or  how  much  they  agree  with  each  item.  In  general,  self-report 
personality  assessment  methods  are  high  in  reliability  and  validity  due  to  the  standardized 
nature  of  administration,  scoring,  and  evaluation  of  the  results  [SCHU90]. 

The  Myers-Briggs  Type  Indicator  (MBTI)  is  a  self-report  personality  survey 
created  by  Katharine  Briggs  and  Isabel  Briggs  Myers  in  the  1920s.  The  MBTI  is  based 
upon  Carl  Jung's  model  of  personality  and  is  the  primary  method  for  measuring  Jungian 
personality  types.  The  MBTI  measures  introversion  and  extroversion  and  is  used  for 
research  purposes  as  well  as  career  counseling.  The  MBTI  requires  several  hours  to 
administer  and  evaluate  and  requires  a  trained  psychological  professional  to  interpret  the 
scores. 

The  Minnesota  Multiphasic  Personality  Inventory  (MMPI)  is  another  frequently 
utilized  personality  assessment  tool.  The  MMPI  determines  personality  traits  of 
hypochondriasis,  depression,  hysteria,  psychopathic  deviate,  masculinity-feminity, 
paranoia,  psychasthenia,  schizophrenia,  hypomania,  and  social  introversion.  The  MMPI 
is  primarily  used  by  clinical  psychologists  as  a  diagnostic  tool  for  assessing  personality 
disorders,  but  is  also  utilized  as  a  vocational  tool.  Unfortunately,  like  the  MBTI,  the 
MMPI  is  extremely  long  to  administer  and  requires  special  training  to  interpret  the 
results. 
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Projective  testing  methods  are  primarily  utilized  for  assessing  disturbed 
individuals.  When  presented  with  an  ambiguous  stimulus,  such  as  a  Rorschach  inkblot, 
the  patient  will  project  personal  needs,  values,  and  fears  onto  the  stimulus  description. 
Projective  techniques  suffer  from  low  reliability  and  validity  due  to  the  subjective  nature 
of  the  result  evaluations  [SCHU90]. 

Behavioral  assessment  procedures  evaluate  a  person's  behavior  to  a  specific 
situation.  Researchers  assessing  the  personality  of  an  entire  group  of  people  primarily 
utilize  this  method.  For  example,  hospital  staff  will  routinely  observe  patient  behavior  in 
order  to  identify  behavioral  trends  in  patients.  This  method  requires  specifically  trained 
observers  and  is  highly  subject  to  observer  bias,  resulting  in  lower  reliability  and  validity 
[SCHU90]. 
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III.  APPARATUS 


A.         NEO  FFI 

The  modeling  of  an  expert  CO  response  to  an  UNREP  evolution  is  theoretically 
possible  if  the  individual  differences  of  each  CO  can  be  ascertained.  Collecting  accurate 
data  about  individual  differences  of  COs  requires  selecting  the  correct  personality 
assessment  tool.  Most  COs  are  limited  on  time  and  relatively  unsupportive  of  academic 
endeavors  that  take  away  from  their  operational  duties.  While  the  assessment  tool  must 
be  highly  reliable  and  valid,  because  of  the  population  being  examined  and  the  purposes 
of  this  research,  it  must  also  be  easy  to  administer,  easy  to  complete  with  minimal  time 
requirements,  and  easy  to  evaluate  with  little  training  required. 

While  the  MMPI  is  a  predominantly  used  objective  test  for  assessing  personality, 
it  is  primarily  used  for  assessing  personality  disorders  [BERN94].  The  MBTI  is  a  widely 
utilized  personality  inventory  implemented  in  career  related  management,  but  requires  a 
trained  psychological  professional  to  administer  and  is  too  time  intensive  for  the  purposes 
of  this  research  [SCHU90].  Though  there  are  a  number  of  objective  tests  designed  to 
measure  a  broad  range  of  personality  variables  in  a  normal  population,  an  increasingly 
popular  choice  is  the  Neo  Personality  Inventory  (NEO  PI-R)  [BERN94].  The  NEO  PI-R 
is  a  prime  choice  for  inventorying  an  expert's  personality  since  it  is  the  predominant 
measure  of  the  five  factor  model  of  personality  [WIDI97].  The  NEO  PI-R  consists  of  240 
statements  to  which  a  person  indicates  an  extent  of  agreement  on  a  5-point  scale.    The 
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NEO  PI-R  is  often  referred  to  as  a  lexical  five  factor  model  since  it  attempts  to  define 
personality  in  natural  language  terms. 

The  "majority  of  academic  psychologists,  increasingly  favor  the  NEO  PI-R  for 
assessment  and  research"  [FURN96].  Furthermore,  substantial  research  exists  regarding 
NEO  PI-R  reliability  and  validity.  Most  important,  the  NEO  PI-R  has  demonstrated 
consistent  convergent  and  discriminant  validity,  as  well  as  indicating  how  alternate 
models  can  be  understood  from  the  perspective  of  the  five  factor  model  [MCCR89]. 
Multiple  studies  have  correlated  established  measures  with  the  NEO  PI-R  to  establish 
overlap,  including  the  Eysenck  Personality  Inventory  and  Myers-Briggs  Type  Indicator 
[FURN96]. 

The  creators  of  the  NEO  PI-R  re-evaluated  the  usefulness  and  applicability  of 
their  personality  assessment  test.  Their  findings  indicated  "far  more  evidence  of  its 
comprehensiveness,  universality,  and  practical  relevance  today  than  when  the  NEO-PI 
was  first  published"  [MCCR97].  The  NEO  Five  Factor  Inventory  (NEO-FFI)  is  a  brief 
60-question  subset  of  the  full  240  question  NEO  PI-R.  The  NEO  PI-R's  additional  length 
allows  for  more  precise  measurement  and  better  false  answer  detection  while  the  NEO- 
FFI  shorter  length  accommodates  a  quicker  administration  time  for  the  participant.  Since 
the  creators  of  the  NEO  PI-R  do  not  envision  any  significant  changes  in  the  structure  of 
the  NEO  PI-R  in  the  near  future,  it  is  a  logical  conclusion  that  there  are  no  major 
revisions  planned  for  the  NEO-FFI. 

The  NEO-FFI  scales  show  correlations  with  the  NEO  PI-R  ranging  from  .75  to  .89 
for  each  of  the  five  factors.  As  subsets  of  the  NEO  PI-R  domain  scales,  the  NEO-FFI 
scales  carry  portions  of  the  demonstrated  validity  of  the  full  scales.   While  the  NEO-FFI 
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scales  are  not  equivalent  to  the  full  scales  of  the  NEO  PI-R,  the  shorter  scales  are 
approximately  85  percent  as  accurate  as  the  full  scales.  In  the  case  of  the  abbreviated 
scales  of  the  NEO  FFI,  some  precision  is  traded  for  speed  and  convenience  [MCCR92]. 

Previous  research  inventoried  five  senior  US  Navy  Surface  Warfare  Officers  for 
individual  differences  to  determine  variability  in  personality  using  the  NEO-FFI  [NPS99]. 
Participants  consisted  of  five  Unites  States  Navy  Commanders,  military  pay  grade  of  O- 
5,  all  of  which  have  been  designated  Surface  Warfare  their  entire  careers.  Four  of  the  five 
had  served  in  Executive  Officer  positions  as  their  last  sea  going  billet,  and  one  had  served 
as  a  Commanding  Officer.  All  five  participants  scored  in  the  low  category  for 
Neuroticism  with  little  variance.  On  average,  the  participants  were  high  in  Agreeableness 
and  Extraversion  and  average  in  all  other  categories.  The  participants  exhibited  large 
variances  in  Openness  and  Agreeableness  scores. 

This  previous  research  justified  the  choice  of  the  NEO-FFI  as  the  tool  to  assess 
the  personality  of  expert  ship  drivers  [NPS99].  All  participants  clearly  understood  the 
standardized  directions  and  had  no  questions.  During  the  pilot  experiment,  the  inventory 
was  easy  to  administer  and  on  average  took  less  than  10  minutes  to  complete.  Evaluation 
of  the  results  required  minimal  time  and  were  very  easy  to  interpret  by  the  researcher  who 
had  no  formal  personality  assessment  training. 


B.         EXPERT  SHIP  HANDLING  SURVEY 

As  part  of  developing  initial  profiles  for  a  VCO  for  COVE,  a  ship  handling 
background  questionnaire  was  utilized.     While  the  questionnaire  was  primarily  for 
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demographic  purposes,  it  did  attempt  to  elicit  participant  opinions  about  how  to  train  a 
junior  ship  driver.  The  questionnaire  was  combined  with  ship  driver  interviews  in  order 
to  determine  the  basic  VCO  framework  [TENN99]. 

Previous  research  surveying  and  analyzing  ship-driving  Commanding  Officers 
highlighted  traits  and  characteristics  that  are  the  same  as  the  traits  exhibited  by  experts 
[TENN99].  Based  upon  this  commonality,  experienced  ship-drivers  were  identified  as 
experts.  Expounding  upon  the  original  research,  this  investigation  also  utilizes  a  survey. 
In  the  early  phases  of  research  development,  it  was  determined  that  a  large  population  of 
expert  participants  was  desired  to  ensure  that  the  full  spectrum  of  ship  driving  styles  was 
approached.  Unfortunately,  because  of  logistical  constraints,  any  form  of  physical 
interview  was  impractical.  Therefore,  the  investigator  decided  to  utilize  a  survey  for  the 
primary  method  of  ship  driving  style  elicitation. 

Survey  questions  were  primarily  based  upon  previous  research  examining 
individual  differences  and  ship  driving  style  [NPS99].  In  the  previous  research,  the 
participating  expert  ship  handlers  were  administered  an  open  ended  interview  regarding 
UNREP.  The  expert  participants  were  encouraged  to  state  what  the  key  factors  were 
when  they  evaluated  novice  ship  handlers.  The  results  of  these  expert  interviews  build 
the  fundamental  core  of  this  research's  survey. 

However,  utilizing  a  questionnaire  for  the  ship  driving  style  elicitation  posed 
challenges  that  required  significant  consideration.  Since  the  survey  was  to  be  performed 
remotely  by  the  ship  handling  evaluator  participants,  the  survey  must  be  extremely  clear 
since  the  researcher  would  not  be  present  to  make  any  clarifications.  The  size  of  the 
population  of  expert  ship-handlers  precluded  qualification  of  a  participant  as  an  expert  for 
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reasons  other  than  experience  and  position.  Also,  the  questionnaire  had  to  be  concise 
since  most  expert  ship  handlers  have  limited  time  to  diverge  from  their  duties  of  running 
a  warship.  The  goal  was  to  maintain  the  expert  ship  handling  survey  completion  time  to 
under  15  minutes. 

The  survey  format  primarily  utilizes  multiple  choice  or  rating  questions  to  elicit 
the  desired  information  from  the  participant.  Rating  questions  were  specifically  chosen 
because  they  produce  an  actual  or  absolute  value  of  the  trait  being  measured.  This 
required  developing  the  rating  scale  with  equal  intervals  with  an  anchor  position.  These 
traits  result  in  rating  questions  being  easier  to  write  and  prone  to  fewer  errors  [GA093]. 

Figure  2  details  the  ratings  utilized  for  the  survey.  The  rating  scale  was 
specifically  developed  to  minimize  respondent  error  and  bias.  The  list  of  possible  choices 
was  set  at  five  since  most  respondents  can  only  distinguish  between  five  to  nine  items 
[GA093].  Furthermore,  the  list  was  maintained  short  in  order  to  reduce  primacy  and 
recency  effects,  effect  where  respondents  are  biased  toward  the  last  few  items  because 
those  are  freshest  in  memory  of  a  long  list  of  items.  The  list  of  possible  rating  responses 
was  always  presented  in  the  same  ascending  order  to  facilitate  proper  understanding  of 
each  rating  and  help  aid  recall. 

Another  primary  concern  with  developing  a  question  involves  avoiding 
inappropriate  questions.  The  expert  ship-handling  domain  is  extremely  sensitive  to 
perceived  right  and  wrong  ways  of  doing  business.  For  example,  this  requires  avoiding 
questions  that  might  require  an  answer  that  is  directly  contrary  to  guidance  or  doctrine. 
Regardless  of  whether  or  not  the  expert  disagrees  with  doctrine,  it  would  be  socially 
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unacceptable  for  the  expert  to  declare  that  he  conducts  business  in  a  contrary  manner.   In 
general  survey  questions  were  developed  to  avoid  the  following  questions  that: 


Are  not  relevant  to  the  evaluation  goals; 

Are  perceived  as  an  effort  to  obtain  biased  or  one  sided  results; 

Cannot  or  will  not  be  answered  accurately; 

Are  not  geared  to  the  respondent's  depth  and  range  of  information, 
knowledge,  and  perceptions; 

Are  not  perceived  by  respondents  as  logical  and  necessary; 

Require  an  unreasonable  effort  to  answer; 

Are  threatening  or  embarrassing; 

Are  vague  or  ambiguous;  or 


Are  unfair 


[GA093]. 


Not  Applicable  (N/A)  -There  is  no  need  to  perform  this  action. 

Applicable  (A)  -  This  is  a  relatively  minor  action  with  large  room  for 
variation  of  execution. 

Somewhat  Important  (SI)  -  An  action  that  must  be  performed  to  have 
a  successful  UNREP  approach,  but  with  some  room  for  variation  of 
execution. 

Important  (I)  -  This  action  must  be  performed  well  to  have  a  successful 
approach  with  little  variation  of  execution. 

Critical  (C)  -  It  is  impossible  to  successfully  complete  an  UNREP 
approach  without  performing  this  action  flawlessly. 


Figure  2:  Rating  scale  utilized  for  the  Ship  Driving  Style  Survey 
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In  addition  to  avoiding  inappropriate  questions,  the  questions  themselves  "must  be 
direct,  orderly,  precise,  logical,  concise,  and  grammatically  correct.  They  must  have 
unity,  coherence,  and  emphasis"  [GA093]. 

C.         EXPERIMENT  PACKAGE 

The  experiment  package  consisted  of  a  single  survey  consisting  of  five 
subsections: 

•  Introduction  and  Background. 

•  Participant  Expert  Ship  handler  Demographic  Questionnaire 

•  Personality  Inventory 

•  Expert  Ship  Handler  Survey 

•  Conclusion  and  Comments. 

The  overall  goal  was  to  maintain  the  total  completion  time  to  less  than  30 
minutes,  because  it  was  a  reasonable  amount  of  time  to  accommodate  completion  of  the 
survey  during  a  lunch  hour  or  other  mealtime  underway  or  in  port.  Furthermore, 
anonymity  and  privacy  were  highly  stressed  to  promote  participation  and  help  elicit 
higher  quality  responses. 

Two  forms  of  the  survey  were  created,  an  Internet  based  survey  and  a  traditional 
paper  based  survey.  The  layout  of  the  traditional  paper  based  survey  utilized  Government 
Accounting  Office  (GAO)  survey  guidelines  for  design.  For  example,  font  size  was 
maintained  at  10-point  type  and  text  was  arranged  into  two  columns  to  promote  ease  of 
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reading  [GA093].  The  survey  was  printed  front  to  back  to  minimize  the  apparent  size  of 
the  document,  reducing  the  likelihood  that  a  potential  expert  ship  handler  would  not 
complete  the  survey  due  to  time  requirements.  Furthermore,  the  survey  was  bound  to 
improve  appearance  and  better  accommodate  the  participant. 

The  Internet  based  survey  was  created  in  order  to  allow  participants  the 
opportunity  to  complete  the  experiment  without  having  to  return  any  package  via  the  mail 
system.  Content  was  exactly  identical  to  the  paper  based  version,  however  some 
formatting  was  changed  to  better  accommodate  viewing  on  a  1 7  inch  computer  monitor. 
The  visual  layout  was  optimized  for  an  800x600  pixel  screen  size.  The  Internet  based 
survey  also  automated  data  collection  by  sorting  the  participants'  responses  into  a 
database,  removing  the  possibility  of  any  data  corruption  by  the  researcher.  This  version 
of  the  survey  was  created  using  Microsoft  FrontPage  2000®  web  development  software 
with  all  code  generated  into  Hyper  Text  Markup  Language  (HTML). 

Prior  to  commencement  of  the  experiment,  two  senior  surface  warfare  officers 
stationed  at  Naval  postgraduate  School  tested  the  Internet  version  of  the  survey  for 
usability  and  functionality.  One  usability  participant  was  a  senior  Navy  Commander, 
military  pay  grade  0-5,  who  had  previously  served  in  the  position  of  Executive  Officer  of 
a  warship  at  sea.  The  other  participant  was  a  senior  Navy  Captain,  military  pay  grade 
0-6,  who  had  previously  served  as  a  Commodore  of  a  squadron  of  warships.  Changes 
regarding  content  of  the  Internet  version  of  the  survey  were  also  made  in  the  paper-based 
version  of  the  survey  to  maintain  continuity  between  the  two  experiment  forms. 

Every  expert  ship  handler  requested  to  participate  in  the  experiment  was  mailed 
the  following  items: 
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•  Cover  letter  requesting  participation  from  either  Commander 
Surface  Naval  Forces  Atlantic  Vice  Admiral  Giffin  (see 
APPENDIX  A),  or  Commander  Surface  Naval  Forces  Pacific  Vice 
Admiral  Moore  (see  APPENDIX  B). 

•  Addendum  to  instructions  detailing  how  to  complete  the  Internet 
based  version  of  the  experiment  survey  (see  APPENDIX  C). 

•  A  paper  based  copy  of  the  experiment  survey  (see  APPENDEK  D). 

•  A  pre-addressed  return  envelope. 

While  every  United  States  Navy  surface  warship  possesses  Internet  capability  while  in 
port  or  at  sea,  every  expert  ship  handler  was  given  the  choice  of  participating  via  either 
the  Internet  based  or  traditional  paper  based  survey  version.  While  possessing  the 
capability,  it  might  not  be  feasible  for  an  expert  ship  handler  to  participate  in  the 
experiment  electronically  because  of  operational  constraints  or  material  maintenance. 
Furthermore,  the  dual  method  of  participation  accounted  for  problems  with  electronic 
participation  and  also  helped  prevent  requested  experts  from  not  participating  because 
they  were  not  comfortable  with  the  method  of  electronic  participation. 
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IV.  METHODOLOGIES 


A.         EXPERT  POPULATION  CANIDATES 

Along  with  deciding  what  to  ask,  evaluators  must  decide  whom  to  ask. 
The  people  questioned  must  have  the  information  the  evaluators  seek,  they 
must  be  readily  identifiable  and  accessible,  they  must  be  willing  and  able 
to  answer,  and  they  must  be  representative  of  the  population  being 
measured. 

[GA093] 

Since  the  purpose  of  this  research  is  to  learn  about  the  relationships  between 
expert  evaluators  and  novices  performing  a  subjective  task,  the  experiment  obviously 
required  experts  in  a  position  to  evaluate  novices.  Because  COVE  is  the  vehicle  for  the 
research,  the  target  population  of  this  research  was  the  expert  ship-handlers  in  a  position 
to  train  and  evaluate  junior  novice  ship-handlers. 

Even  though  the  CO  is  the  ultimate  person  responsible  for  all  ship  operations  and 
sets  the  tone  for  the  conduct  of  all  operations,  he  is  not  necessarily  the  only  instructor  and 
evaluator.  In  most  circumstances,  even  during  an  UNREP,  the  Executive  Officer  (XO) 
also  plays  a  vital  role  in  instruction  and  evaluation  of  junior  novice  ship-handlers  by 
augmenting  the  CO  as  an  additional  coach  or  evaluator.  While  not  as  common,  a 
Department  Head  (DH)  is  an  occasional  additional  mentor  to  the  junior  ship-handler  and 
sometimes  provides  input  to  the  CO  for  evaluation  of  the  JO. 
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The  experiment  primarily  targeted  a  sample  of  COs  and  XOs  currently  serving 
aboard  warships  in  the  fleet.  DH  survey  participation  was  also  accepted  if  the  DH  was  a 
recognized  ship-handling  expert  by  his  CO  or  XO.  These  experts  were  selected  because 
they  are  currently  performing  the  analyzed  task  and  are  most  familiar  with  the  current 
doctrine  and  equipment  utilized  in  the  fleet.  In  addition  to  being  a  CO,  XO,  or  DH,  the 
expert  candidate  must  be  serving  on  a  class  of  warship  that  conducts  UNREPs  as  the 
approach  vessel.  This  resulted  in  the  exclusion  of  small  craft  such  as  mine  hunters  and 
coastal  patrol  craft.  Furthermore,  while  tankers  occasionally  perform  an  UNREP  as  the 
approach  vessel,  non-Navy  personnel  who  might  not  possess  the  same  background  as  the 
targeted  expert  population  usually  operate  them. 

Based  upon  the  class  of  ship  criteria,  there  were  a  combined  total  of  171  eligible 
warships  between  the  Atlantic  and  Pacific  naval  forces.  Since  every  US  Naval  warship 
has  both  a  CO  and  an  XO,  there  were  a  total  of  342  potential  ship-handling  experts  to 
sample  from.  While  a  larger  sample  size  of  ship-handling  experts  will  be  a  better 
approximation  of  the  total  population  of  ship-handling  experts,  a  minimum  of  30 
experiment  participants  is  required  to  satisfy  the  Central  Limit  Theorem  statistical  rule  of 
thumb  [DEV095]. 


B.         SURVEY  ADMINISTRATION 

Experiment  packages  were  assembled  and  mailed  via  United  States  Post  Office 
First  Class  delivery  to  all  342  ship-handling  expert  candidates.  The  candidates  were 
allowed  approximately  three  weeks  until  the  beginning  of  July  2000  to  complete  either 
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the  web  based  or  paper  based  survey.  The  administration  period  was  selected  to 
accommodate  mail  time  both  to  and  from  the  ship  as  well  as  an  adequate  time  to  review 
and  complete  the  survey. 

If  the  web-based  version  was  completed,  the  experiment  participant  was  instructed 
to  not  return  the  paper-based  version.  If  the  participant  completed  the  paper-based 
version,  the  data  was  recorded  utilizing  the  web-based  survey  after  verifying  that  the 
paper-based  survey  was  not  a  duplicate  of  an  already  submitted  web-based  survey.  This 
manual  conversion  of  survey  format  was  performed  to  accommodate  automatic  data 
collation  and  analysis. 


C.         ANALYSIS 

All  raw  survey  results  were  compiled  into  a  single  Microsoft  Access*  database. 
The  database  file  was  then  exported  into  a  Microsoft  Excel®  spreadsheet.  Once  in  Excel 
format,  the  raw  personality  scores  for  Neuroticism  (N),  Extraversion  (E),  Openness  (O), 
Agreeableness  (A),  and  Conscientiousness  (C)  were  computed. 

Questions  from  the  ship-handling  evaluation  section  of  the  survey  were  classified 
as  one  of  six  distinct  types  consisting  of: 


INTERACTION  questions  measuring  the  expert  ship-handler's 
preference  for  interaction  between  the  novice  JO  and  other  entities. 
Other  entities  could  consist  of  the  other  members  of  the  bridge 
team,  the  replenishment  ship,  or  the  expert  ship-handler  himself. 

ANALYTIC  INPUT  questions  measuring  the  expert  ship-handler's 
preference  for  the  type  of  rational  based  decision  information  that  a 
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junior  novice  ship-handler  should  use.  Examples  of  Analytic  Input 
are  rules  of  thumb,  numerical  data  from  ship's  sensors,  and 
standard  operating  procedures, 

SENSORY  INPUT  questions  measuring  the  expert  ship-handler's 
preference  for  the  type  of  instinctual  based  decision  information 
that  a  junior  novice  ship-handler  should  use.  Examples  of  Sensory 
Input  are  visual  approximations  for  range,  non-numerical  or  non- 
calculation  based  rules  of  thumb,  and  kinesthetic  approximations 
for  weather  forces. 

COMMUNICATION  questions  measuring  the  expert  ship- 
handler's  preference  for  the  type  of  communications  the  junior 
novice  ship-handler  should  use.  Examples  of  Communication  are 
internal  and  external  communications  circuits. 

UNREP  STYLE  questions  measuring  the  general  expert  ship- 
handler's  approach  to  UNREP  and  what  he  expects  of  the  junior 
novice  ship-handler.  UNREP  Style  questions  also  include  the 
expert  ship-handler's  general  interpretation  of  UNREP  guidance 
and  doctrine. 


The  average  response  for  each  of  the  six  groups  was  calculated  for  each  expert 
ship-handling  participant.  All  data  was  then  converted  into  an  input  file  for  the  ARC 
software  package,  a  menu  driven  statistical  analysis  tool  developed  at  the  University  of 
Minnesota  for  applied  regression. 

Once  the  data  package  was  loaded  into  ARC,  a  statistical  summary  of  the  data 
package  was  created.  The  statistical  summary  contains  information  such  as  mean  values, 
minimum  and  maximum  values,  median  values,  and  standard  deviations.  Furthermore, 
the  statistical  summary  contains  a  matrix  of  correlation  values  between  the  different 
variables  of  the  data  package.  It  is  from  this  matrix  that  significant  correlations  were 
retrieved  for  discussion. 
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V.  RESULTS  AND  DISCUSSION 


A.         PARTICIPANT  DEMOGRAPHICS 

Of  the  342  ship-handling  experts  polled,  136  experts  participated  in  the  survey,  of 
which  35  participated  via  the  Internet.  Eight  surveys  were  incomplete  and  not  used  in  the 
data  analysis.  One  survey  was  completed  by  a  participant  who  did  not  meet  the  criteria  of 
a  ship-handling  expert  as  defined  for  this  experiment.  Eleven  surveys  were  returned  too 
late  to  be  included  in  the  data  analysis  package.  The  resulting  data  analysis  package 
consisted  of  a  total  of  1 16  surveys. 

At  the  time  of  the  survey  administration,  65  survey  participants  were  serving  in 
the  CO  billet  and  48  were  serving  in  the  XO  billet,  and  2  were  serving  in  the  DH  billet. 
Of  all  participants,  only  two  ship-handling  experts  are  female.  Table  1  further 
summarizes  some  demographical  information  of  the  116  analyzed  ship-handling  expert 
participants. 

On  average,  the  ship-handling  expert  participants  had  served  under  eight  different 
COs  during  their  career.  Furthermore,  the  average  participant  had  performed  between  50 
and  100  UNREPs  during  their  career  with  a  single  ship-handling  expert  who  had 
performed  over  300  UNREPs.  Eight  participants  were  aviators  and  have  not  been  surface 
warfare  qualified  for  their  entire  careers.  Major  ship  classes  represented  consisted  of: 
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•  Aircraft  carriers,  including  both  nuclear  powered  (CVN)  and  non- 
nuclear  powered  (CV)  types, 

•  Guided  missile  cruisers  (CG), 

•  Destroyers,  including  both  guided  missile  (DDG)  and  non-guided 
missile  (DD)  types, 

•  Guided  missile  frigates  (FFG), 

•  Transport  ships  including  amphibious  assault  ships  (LHA/LHD), 
dock  landing  ships  (LSD),  tank  landing  ships  (LST),  and 
amphibious  transport  dock  ships  (LPD), 

•  Other  warships  not  classified  above. 


Figure  3  is  a  histogram  that  delineates  the  frequency  of  ship  types  for  the 
participating  expert  ship-handlers.  Figure  4  is  a  histogram  that  describes  the  participant 
billet  distribution  between  observed  ship  types. 


Average 
Observed 

Minimum 
Observed 

Maximum 
Observed 

Standard 
Deviation 

Age  (Years) 

40.4 

33 

50 

4.1 

Rank 

Commander 

Lieutenant 
Commander 

Captain 

N/A 

Years  Of  Service 
As  An  Officer 

18.1 

11 

28 

3.9 

Years  Of  Service 
At  Sea 

10.5 

5 

22 

3.1 

Table  1 .  Demographic  Information  for  Ship-Handling  Expert  Participants. 
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Figure  3.  Frequency  of  Participant  Ship-Handling  Expert  Ship  Types 


CG 


CV/CVN  DD/DDG 


FFG  Transport 


Figure  4.  Frequency  of  Billet  Distribution  Among 
Participant  Ship-Handling  Expert  Ship  Types 
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B. 


PERSONALITY  INVENTORY  RESULTS 


Table  2  provides  a  statistical  summary  of  the  observed  expert  ship-handler  XEO- 
FFI  results.  Figure  5  highlights  the  personality  differences  between  the  average 
participant  expert  ship-handler  and  the  average  over  18  year-old  adult  participants  as 
defined  within  the  XEO-FFI  manual  [MCCR92].  Means  and  standard  deviations  are 
contrasted  for  each  personality  trait.  Notable  differences  are  evident  in  Xeuroticism, 
Extraversion,  and  Conscientiousness. 


Neuroticisra 
(N) 

Extraversion 
(E) 

Openness 
(O) 

Agreeableness 
(A) 

Conscientiousness 
(C) 

Average  ; 

10.91 

34.69 

27.36 

32.67 

38.66 

Min 

0.00 

15.00 

15.00 

13.00 

24.00 

Max 

31.00 

46.00 

43.00 

46.00 

48.00 

Mode 

6.00 

33.00 

27.00 

34.00 

36.00 

Median 

10.50 

35.00 

27.00 

33.00 

39.00 

Table  2.  Statistical  Summary  For  Participant  Expert  Ship-Handler  XEO-FFI  Results 

Table  3  delineates  the  personality  differences  amongst  the  participant  expert  ship- 
handlers.  Personality  scores  were  relatively  consistent  with  the  exception  of  carrier 
expert  ship-handlers.  On  average,  carrier  ship-handling  experts  are  significantly  lower  in 
Xeuroticism  and  higher  in  Openness  and  Agreeableness.  These  personality  differences 
could  be  related  to  the  demographic  difference  between  carrier  ship-handlers  and  other 
surface  ship-handlers  since  all  carrier  COs  and  XOs  are  aviators.  The  only  other 
demographic  group  to  show  slight  deviations  from  the  whole  was  the  cruiser  ship- 
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handling  experts  who  typically  scored  slightly  lower  in  Neuroticism  and  higher  in 
Extraversion  than  the  average  participant  expert  ship-handler. 


50- 
45  - 
40  - 
35  - 
30  - 
25  - 
20  - 
15" 
10  - 
5  - 
0  - 


Q  Expert  Ship-Handlers  CI  Typical  Adult 


Figure  5.  A  Comparison  of  Observed  Expert  Ship-Handler  Means 

and  Standard  Deviations  to  Typical  Adults  As  Defined  By  The 

NEO-FFI  Professional  Manual. 


CG 

CV/CVN 

DD/DDG 

FFG 

Transport 

Neuroticism 

9.53 

6.60 

10.03 

13.96 

11.35 

Extraversion 

37.00 

39.20 

34.33 

33.04 

33.69 

Openness 

26.42 

32.40 

27.80 

26.96 

26.39 

Agreeableness 

29.74 

39.00 

32.72 

32.96 

33.19 

Conscientiousness 

40.11 

42.20 

38.69 

38.11 

37.23 

Table  3.  Mean  Participant  Expert  Ship-Handler  NEO-FFI 
Results  For  Each  Major  Participant  Ship  Class 
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In  order  to  compare  a  participant's  raw  personality  score  to  an  average  distribution 
of  adults,  the  raw  score  is  converted  into  a  standardized  classification  group.  The 
standardized  personality  classification  groups  consist  of  very  low,  low.  average,  high,  and 
very  high.  Ranges  for  each  of  the  classifications  depend  upon  which  score  is  being 
classified.  Table  4  contains  the  standardized  values  for  each  personality  classification 
range  as  defined  in  the  NEO-FFI  manual  [MCCR92]. 


Neuroticism 

Extraversion 

Openness 

Agreeableness 

Conscientiousness 

Very  Low 

0-6 

0-18 

0-  18 

0-24 

0-25 

Low 

7-13 

19-24 

19-23 

25-29 

25-30 

Average 

16-21 

25-30 

24-30 

30-34 

31  -37 

High 

22-29 

31  -  36 

31  -36 

25-40 

38-43 

Verv-  High 

30-50 

37-  50 

37-50 

41  -50 

44-50 

Table  4.  NEO-FFI  Standardized  Raw  Score  Classification  Ranges 

Figure  6  is  a  histogram  depicting  the  frequency  of  Neuroticism  scores  amongst  the 
participant  expert  ship-handlers.  The  distribution  is  positively  skewed  indicating  that  a 
majority  of  the  participant  expert  ship-handlers  possess  low  Neuroticism.  Figure  7  is  a 
histogram  that  illustrates  the  distribution  of  Neuroticism  classifications  amongst  the 
participant  expert  ship-handlers.  Figure  7  confirms  that  97%  of  the  participants  possess 
average  or  lower  than  average  Neuroticism  as  defined  in  Table  4  [MCCR92]. 
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Neuroticism  Raw  Score 


Figure  6.  Frequency  of  Neuroticism  Raw  Scores 
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Figure  7.  Distribution  of  Standardized  Neuroticism  Scores 
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Figure  8  is  a  histogram  highlighting  the  distribution  of  participant  expert  ship- 
handler  Extraversion  scores.  Figure  8  illustrates  a  negatively  skewed  distribution 
indicating  that  most  participants  possess  iarge  scores  for  Extraversion.  Figure  9,  a 
histogram  delineating  the  breakdown  of  the  Extraversion  raw  scores,  communicates  that 
most  participant  expert  ship-handlers  were  higher  than  average  in  Extraversion. 
Approximately  80%  of  the  participant  expert  ship-handlers  possess  a  high  or  very  high 
Extraversion  personality  characteristic. 

Figure  10  is  a  histogram  that  displays  the  frequency  of  the  observed  Openness  raw 
scores.  Unlike  Extraversion  and  Neuroticism,  the  participant  expert  ship-handlers  appear 
to  possess  a  symmetric  distribution  of  Openness  personality  characteristics.  Figure  1 1 , 
the  distribution  of  Openness  classifications  amongst  the  participants,  appears  to  center 
around  the  average  with  an  approximately  normal  distribution.  36%  of  all  participant 
expert  ship-handlers  possess  an  average  Openness  personality  characteristic  while  only 
28%>  possess  a  low  characteristic  and  only  22%  possess  a  high  Openness  characteristic. 

Figure  12  is  a  histogram  illustrating  the  frequency  of  Agreeableness  raw  scores. 
Similar  to  Openness.  Agreeableness  also  appears  to  be  symmetrically  distributed  amongst 
the  participant  expert  ship-handlers.  However,  as  Figure  13  details,  the  majority  of 
participants  possess  an  average  level  of  Agreeableness  while  only  27%  possess  a  lower 
than  average  level  and  only  32%  possess  a  higher  than  average  Agreeableness  personality 
trait. 
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Figure  8.  Frequency  of  Extraversion  Raw  Scores 
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Figure  9.  Distribution  of  Standardized  Extraversion  Scores 
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Figure  10.  Frequency  of  Openness  Raw  Scores 
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Figure  1 1 .  Distribution  of  Standardized  Openness  Scores 
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Figure  12.  Frequency  of  Agreeableness  Raw  Scores 
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Figure  13.  Distribution  of  Standardized  Agreeableness  Scores 
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Figure  14  is  a  histogram  that  displays  the  frequency  of  Conscientiousness  raw 
scores.  Conscientiousness  appears  to  have  a  slightly  negatively  skewed  distribution 
amongst  the  participant  expert  ship-handlers.  Figure  15  showing  the  distribution  of  raw 
score  classifications  for  Conscientiousness  shows  that  88%  of  all  participants  possess  an 
average  to  very  high  Conscientious  personality  trait. 
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Figure  14.  Frequency  of  Conscientiousness  Raw  Scores 
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Fisure  15.  Distribution  of  Standardized  Conscientiousness  Scores 
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C.         SHIP  HANDLING  EVALUATION  SURVEY  RESULTS 

Figure  16  is  a  histogram  that  summarizes  the  overall  participant  expert  ship- 
handler's  preference  for  interaction.  In  general,  the  experts  viewed  the  novice  JO's 
ability  to  interact  with  other  entities  as  a  relatively  import  criteria  for  their  evaluation  of 
the  JCTs  performance.  65%  of  all  respondents  felt  that  how  the  JO  interacts  is  at  least  an 
important  criterion  for  evaluation.  Furthermore,  these  experts  desired  to  coach  their 
novices  through  the  evolution  via  continuous  input  and  feedback  and  their  primary 
measure  of  interaction  is  how  well  the  novice  JO  maintained  close  verbal  communication 
with  the  expert  ship-handler  coach. 
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Figure  16.  Distribution  of  Interaction  Responses 
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Figure  1 7  is  a  histogram  that  summarizes  the  overall  participant  expert  ship- 
handlers'  preference  for  communication.  The  majority  of  experts  did  not  feel  that  the 
novice  JO's  personal  ability  to  communicate  with  the  Replenishment  ship  was  important 
to  their  evaluation  of  the  JO's  performance.  Most  expert  ship-handlers  feel  that  someone 
other  than  the  novice  JO  performing  the  UNREP  should  handle  personal  communications 
between  the  approach  ship  and  replenishment  ship.  Only  17%  of  all  participants 
expressed  communication  as  an  important  criteria  for  UNREP  performance  evaluation. 
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Figure  17.  Distribution  of  Communication  Responses 

Figure  18  summarizes  the  participant  expert  ship-handlers'  preference  for 
analytical  input.  In  general  most  participant  expert  ship-handlers  believe  that  a  must  be 
able  to  efficiently  receive  and  process  analytical  information.  While  all  participants 
believed  that  the  novice  JO's  ability  is  at  least  somewhat  important,  67%  of  all 
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respondents  felt  that  analytical  input  is  at  least  an  important,  if  not  critical  component  of 
evaluating  the  novice's  performance. 
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Figure  18.  Distribution  of  Analytical  Input  Responses 

Figure  19  summarizes  the  participant  expert  ship-handlers'  preference  for  the 
novice  JO  to  understand  and  efficiently  process  sensory  input.  All  participants  believed 
that  the  novice  JO's  ability  to  demonstrate  an  understanding  of  sensory  input  was  at  least 
somewhat  important  to  the  expert's  overall  evaluation.  35%  of  all  participant  expert 
ship-handlers  view  reaction  to  sensory  information  as  a  critical  component  of  a  successful 
UNREP  and  use  the  JO's  response  to  sensory  information  as  a  major  element  of  UNREP 
performance  evaluation. 
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Figure  19.  Distribution  of  Sensory  Input  Responses 

Figure  20  is  a  histogram  that  provides  a  breakdown  of  how  each  individual 
participant  expert  ship-handler  approaches  UNREP.  Figure  20  displays  a  uni-modal 
symmetric  distribution  with  the  majority  of  participant  expert  ship-handlers  taking  an 
attitude  towards  UNREP  that  is  neither  too  flexible  nor  too  strict.  In  general,  most 
evaluators  allow  some  deviations  from  their  execution  preferences  by  the  novice  JO. 
Only  7%  require  the  novice  to  perform  the  evolution  exactly  as  the  expert  desires  while 
only  5%  of  all  participant  experts  allow  the  JO  to  perform  the  UNREP  in  any  safe 
manner. 

The  participant  expert  ship-handlers  who  tended  towards  a  looser  UNREP  style 
placed  less  emphasis  on  time  to  perform  the  approach  as  a  criterion  for  performance 
evaluation.  In  contrast,  those  experts  who  possess  a  more  rigid  UNREP  style  place  more 
emphasis  on  time  as  an  evaluation  criterion.      Regardless  of  UNREP  style,  most 
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participant  expert  ship-handlers  believe  that  as  experience  increases,  time  to  complete  the 
evolution  will  decrease. 

Approximately  50%  of  all  participant  expert  ship-handlers  believe  that  UXREP 
documentation  and  doctrine  provides  instruction  that  must  be  strictly  adhered  to.  The 
other  50%  of  participant  expert  ship-handlers  interpret  UXREP  documentation  and 
doctrine  as  guidance  that  provides  a  flexible  framework  for  execution. 
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Figure  20.  Distribution  of  UXREP  Styles 


D. 


OBSERVED  CORRELATIONS 


Table  5  contains  the  intercorrelations  observed  between  the  five  personality 
factors  measured  by  the  XEO-FFI.  Table  6  contains  the  average  intercorrelations  for  the 
XEO-FFI  [MCCR92].  In  general,  the  personality  traits  were  more  intercorrelated  for  the 
participant  expert  ship-handlers  than  for  the  average  XEO-FFI  participant.    The  only 
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observed  exceptions  where  the  participants  possessed  lower  than  average  intercorrelations 
were  Neuroticism  /  Conscientiousness,  Extraversion  /  Openness,  and  Agreeableness  / 
Conscientiousness. 


N 

E 

0 

A 

C 

Neuroticism 

-0.55 

-0.10 

-0.44 

-.37 

Extraversion 

0.16 

0.31 

0.38 

Openness 

0.09 

-0.04 

Agreeableness 

0.09 

Table  5.  Observed  NEO-FFI  Intercorrelations  for 
Participant  Expert  Ship-Handlers. 


N 

E 

O 

A 

C 

Neuroticism 

-0.21 

-0.02 

-0.25 

-.53 

Extraversion 

0.40 

0.04 

0.27 

Openness 

0.02 

-0.02 

Agreeableness 

0.24 

Table  6.  NEO-FFI  Intercorrelations  for 
Average  Adults  from  the  NEO-FFI  Manual  [MCCR92]. 
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In  general  statistics,  a  large  correlation  exists  if  the  correlation  coefficient  (p)  is 
greater  than  or  equal  to  0.8  and  a  small  correlation  exists  if  p  is  less  than  or  equal  to  0.5. 
However,  the  intricacies  of  human  behavior  make  analyzing  real  people  more  art  than 
science.  Single  sample  t-Test  with  a  probability  of  type  I  error  (a)  equal  to  0.10  indicate 
that  correlations  with  p  greater  than  0.10  are  significant  for  the  participant  expert  ship- 
handler  sample  [DEV095].  Therefore,  for  the  purposes  of  this  research,  any  correlation 
greater  than  or  equal  to  0.10  was  identified  as  a  potentially  significant  correlation. 

Figure  21  graphically  displays  the  calculated  correlations  between  observed  expert 
ship-handler  Neuroticism  values  and  measured  UNREP  evaluation  criteria.  In  general, 
correlations  were  small  in  magnitude.  All  observed  correlations  were  negative  with  the 
exception  of  the  correlation  between  Neuroticism  and  Communication,  which  was  only 
slightly  positive. 

Figure  22  contains  information  describing  the  calculated  correlations  between 
expert  ship-handler  Extraversion  values  and  measured  UNREP  evaluation  criteria.  In 
contrast  with  the  Neuroticism  correlations,  most  Extraversion  correlations  were  positive 
with  the  exception  of  the  correlation  between  Extraversion  and  UNREP  style.  The  only 
apparent  significant  correlation  was  observed  between  Extraversion  and  Sensory  Input  (p 
=  .18).  This  relationship  could  possible  be  explained  by  Sensory  Input  satisfying  the 
extroverts  need  for  stimulation  from  his  surroundings  [MCCR92]. 
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Figure  21.  Correlations  between  Neuroticism  and  UNREP 
Evaluation  Criteria  for  Participant  Expert  Ship-Handlers 
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Figure  22.  Correlations  between  Extraversion  and  UNREP 
Evaluation  Criteria  for  Participant  Expert  Ship-Handlers 
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Figure  23  highlights  the  correlations  calculated  between  the  observed  expert  ship- 
handler  Openness  personality  characteristic  and  the  measured  UNREP  evaluation  criteria. 
All  calculated  correlations  for  Openness  were  positive.  Furthermore,  all  correlations 
were  only  slightly  positive  with  the  exception  of  the  correlation  between  Openness  and 
Analytical  Input  (p  =  .16)  and  Openness  and  UNREP  Style  (p  =  .16).  This  result  might 
be  related  to  the  modest  association  between  Openness  and  measured  intelligence  since 
Analytical  Input  is  more  calculation  based.  Therefore,  it  is  possible  that  an  expert  that  is 
higher  in  Openness  would  tend  to  more  analytical  or  cognitive  intensive  methods. 
However,  it  must  be  clear  that  Openness  is  not  a  measure  of  analytical  sense  or  actual 
intelligence  [MCCR92]. 

Figure  24  delineates  the  correlations  calculated  between  the  observed  expert  ship- 
handler  Agreeableness  personality  characteristic  and  the  measured  UNREP  evaluation 
criteria.  All  calculated  correlations  were  positive  with  the  exception  of  the  correlation 
between  Agreeableness  and  Communication,  which  was  slightly  negative.  The  only 
apparent  significant  correlation  was  observed  between  Agreeableness  and  UNREP  Style 
(p  =  .16).  A  potential  explanation  for  this  correlation  is  that  the  highly  agreeable  person 
is  sympathetic  to  others  and  eager  to  help,  resulting  in  an  UNREP  style  that  is  more 
malleable  to  the  trainee  novice  [MCCR92].  The  expert  who  scores  high  in  Agreeableness 
sees  the  UNREP  as  a  training  evolution  for  the  novice  and  therefore  has  a  less  rigid 
UNREP  style  resulting  in  greater  variability  and  less  structure  in  order  to  let  the  novice 
learn  from  exploration. 
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Figure  23.  Correlations  between  Openness  and  UNREP 
Evaluation  Criteria  for  Participant  Expert  Ship-Handlers 
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Figure  24.  Correlations  between  Agreeableness  and  UNREP 
Evaluation  Criteria  for  Participant  Expert  Ship-Handlers 
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Figure  25  graphically  displays  the  correlations  calculated  between  the  observed 
participant  expert  ship-handler  Conscientiousness  personality  trait  and  the  measured 
UNREP  evaluation  criteria.  Similar  to  the  Openness  correlations,  all  calculated 
Conscientiousness  correlations  were  positive.  However,  two  significant  correlations 
existed  between  Conscientiousness  and  Analytical  Input  (p  =  .17)  and  Conscientiousness 
and  Sensory  Input  (p  =  .39).  The  correlation  between  Conscientiousness  and  Sensory 
Input  was  the  highest  observed  over  the  entire  experiment.  Experts  who  score  high  in 
Conscientiousness  are  usually  purposeful  and  strong  willed  and  are  often  associated  with 
academic  and  occupational  achievement  [MCCR92].  These  facts  can  be  directly  linked 
to  the  expert  ship-handler's  priority  to  properly  processing  all  forms  of  information  input. 
The  highly  conscientious  expert  will  not  be  able  to  ignore  any  information  that  may 
affect  the  UNREP,  regardless  of  source. 
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Figure  25.  Correlations  between  Conscientiousness  and  UNREP 
Evaluation  Criteria  for  Participant  Expert  Ship-Handlers 


60 


Correlations  were  also  calculated  to  determine  if  there  was  any  relationship 
between  the  participant  expert  ship-handlers'  NEO-FFI  personality  traits  and  Tenney's 
VCO  profiles  [TENN99].  Figure  26  illustrates  the  relationships  between  the  measured 
participant  personality  traits  and  Tenney's  Passive  VCO  profile.  Significant  correlations 
existed  between  the  Passive  VCO  profile  and  the  Agreeableness  (p  =  -.25)  and 
Conscientiousness  (p  =  -.11)  personality  traits.  In  direct  contrast,  Tenney's  Proactive 
VCO  profile  correlated  with  Neuroticism  (p  =  -.20),  Extraversion  (p  =  .13),  and  Openness 
(p  =  -.1 1).  Figure  27  illustrates  these  correlations. 
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Figure  26.  Calculated  Correlation  Between  Observed  Participant  Expert 

Ship-Handler  NEO-FFI  Personality  Traits  and 

Tenney's  Virtual  Commanding  Officer  Passive  Profile. 
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Figure  27.  Calculated  Correlation  Between  Observed  Participant 

Expert  Ship-Handler  NEO-FFT  Personality  Traits  and 

Tenney's  Virtual  Commanding  Officer  Proactive  Profile. 


E.         DISCUSSION 

Due  to  the  assistance  of  Vice  Admiral  Giffin  and  Vice  Admiral  Moore,  40  percent 
of  all  expert  ship-handlers  queried  responded  to  the  survey.  This  relatively  high 
participation  rate  greatly  increased  the  quality  of  the  data  package  by  producing  relatively 
normal  distributions  for  all  areas  analyzed.  Indications  are  that  the  sample  of  participant 
expert  ship-handlers  is  a  decent  approximation  of  the  expert  ship-handling  population. 
Furthermore,  the  wide  range  of  responses  created  a  broader  depth  of  elicitated  ship- 
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handling  knowledge  that  will  directly  translate  into   a  better  database  for  COVE 
programmers. 

Analysis  of  the  participant  expert  ship-handler  data  package  indicates  that  the 
surveyed  experts  possess  some  significant  differences  from  the  average  adult.  In  general, 
experts  were  lower  in  Neuroticism,  and  higher  in  Extraversion  and  Conscientiousness.  It 
is  clear  that  whenever  modeling  an  expert  ship-handler,  close  attention  needs  to  be 
focused  towards  the  expert's  level  of  Neuroticism,  Extraversion,  and  Conscientiousness 
to  ensure  that  the  expert  behaves  in  a  manner  that  is  consistent  with  expertise.  Future 
research  will  verify  that  these  personality  characteristics  apply  to  all  experts  regardless  of 
the  domain  of  expertise. 

It  is  important  to  note  that  there  is  still  room  for  variability  in  an  expert  model's 
personality.  While  the  means  for  participant  expert  ship-handler  levels  of  Neuroticism, 
Extraversion,  and  Conscientiousness  were  significantly  different  from  the  average  adult 
population,  Figure  5  illustrates  that  the  standard  deviations  for  the  participant  expert 
ship-handlers'  personality  traits  were  very  close  to  the  average  adult  standard  deviations. 
This  indicates  that  experts  can  vary  in  personality  just  as  much  as  the  average  person 
does,  just  within  a  different  range. 

In  addition  to  the  model  of  expert  personality,  close  attention  should  be  focused 
towards  the  dynamic  of  expert  evaluation  of  a  novice.  All  ship-handling  experts  are 
going  to  evaluate  novice  performance  based  upon  a  unique  combination  of  evaluation 
criteria.  The  evaluation  criteria  create  a  performance  checklist  that  the  novice  is 
evaluated  against  and  will  consist  of  a  combination  of  Interaction  Responses, 
Communication  Responses,  Analytical  Input  Responses,  Sensory  Input  Responses,  and 
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the  evaluator's  UNREP  Style.  The  emphasis  and  priority  of  this  novice  report  card  is 
related  to  the  evaluator's  individual  personality.  Analytical  Input  will  be  stressed  if  the 
expert  evaluator  is  relatively  high  in  either  Openness  or  Conscientiousness.  Sensory 
Input  will  be  a  critical  criterion  if  the  expert  possesses  a  relatively  large  amount  of 
Extraversion  or  Conscientiousness.  Finally  UNREP  style  will  be  a  major  performance 
criterion  if  the  expert  ship-handler  performing  the  evaluation  scores  high  in  Openness  or 
Agreeableness. 

Analysis  of  the  data  package  also  indicates  that  there  are  links  between  Tenney's 
VCO  profiles  and  the  model  of  expert  personality  proposed  in  this  research.  Experts  who 
are  relatively  low  in  Agreeableness  and  Conscientiousness  will  tend  towards  a  more 
passive  style  of  ship-handling  as  defined  by  Tenney's  previous  work.  Furthermore,  and 
in  direct  contrast,  expert  ship-handlers  who  display  relatively  high  Extraversion  and 
relatively  low  Neuroticism  and  Openness  will  tend  towards  Tenney's  proactive  VCO 
profile. 

With  the  proper  composition  of  personality  traits,  and  proper  combination  of 
evaluation  criteria  and  overall  ship  driving  style,  a  relatively  robust  model  of  an 
experienced  evaluator  of  ship-handling  is  developed.  The  model  is  robust  because  it 
covers  a  relatively  wide  range  of  personality  inputs  resulting  in  a  potentially  wide  range 
of  unique  prototype  expert  ship-handlers.  With  proper  development,  the  model  born  from 
this  research  could  potentially  grow  into  an  accurate  representation  of  human  mentoring. 
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VI.  CONCLUSIONS 


A.         SUMMARY  OF  WORK 

Understanding  how  experts  evaluate  performance  of  tasks  with  large  degrees  of 
variability  is  a  difficult  undertaking.  Just  like  the  tasks  themselves,  the  evaluation  does 
not  always  have  the  same  clear-cut  criteria  for  each  expert  performing  the  evaluation. 
Often,  critical  evaluation  criteria  are  as  individualistic  as  the  evaluating  individual. 

The  primary  purpose  of  this  thesis  was  to  assess  the  relationship  between  an 
expert  evaluator's  personality  and  the  evaluation  criteria  the  expert  employed.  COVE's 
UNREP  trainer,  currently  under  development  by  NAWC-TSD,  provided  the  ideal  vehicle 
for  this  research  since  UNREP  is  a  highly  dynamic  and  challenging  task  performed  by  a 
novice  whose  performance  evaluation  is  often  a  combination  of  unique  objective  and 
subjective  criteria  dependent  upon  the  evaluating  expert. 

In  order  to  obtain  information  about  the  relationship  between  evaluation  and 
personality,  a  review  was  performed  on  various  topics  including  facets  of  personality, 
decision-making,  individual  differences,  differences  between  experts  and  novices,  and 
personality  measurement.  This  review  was  a  crucial  step  for  selecting  the  proper 
personality  assessment  tool.  Next,  an  experiment  package  was  developed  utilizing  a 
questionnaire  format  in  order  to  elicit  information  about  personality,  ship-handling  style, 
and  novice  performance  evaluation  by  a  ship-handling  expert. 
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After  the  experiment  package  was  developed,  the  survey  was  administered  to 
ship-handling  experts  resident  in  the  fleet.  Support  from  both  Commander  Surface  Naval 
Forces  Atlantic,  Vice  Admiral  Giffin,  and  Commander  Surface  Naval  Forces  Pacific, 
Vice  Admiral  Moore  was  critical  in  ensuring  adequate  participation  by  busy  ship- 
handling  experts.  With  their  help,  116  experts  participated  in  the  survey  either  via  a 
paper  based  or  web  based  version. 

After  the  end  of  the  data  collection  period,  the  participant  surveys  were  collated 
into  a  data  analysis  package.  The  data  was  analyzed  for  statistical  summary  information 
such  as  means,  minimum  values,  maximum  values,  mode,  median,  and  standard 
deviations.  The  data  was  also  analyzed  for  significant  correlations  existing  between 
participant  expert  ship-handler  personality  traits  and  participant  expert  ship-handler 
evaluation  criteria  and  ship  driving  style.  Correlations  were  also  analyzed  between 
personality  and  Tenney's  VCO  profiles. 

Understanding  performance  evaluation  is  a  goliath  work  in  progress.  Results  of 
this  research,  while  beginning  to  explore  the  gray  areas  of  variability  in  evaluation,  will 
provide  direct  enhancement  to  the  training  potential  of  COVE  by  increasing  the  fidelity  of 
COVE's  ITS.  The  insights  gained  from  the  expert  ship-handler  personality  traits  will 
increase  the  social  realism  of  the  ITS  and  therefore  improve  the  manner  of  feedback  given 
to  the  trainee  since  the  ITS  can  now  respond  in  a  more  human  manner.  Furthermore,  the 
compilation  of  critical  evaluation  criterion  and  its  relationship  to  personality  will  enhance 
the  task  training  ability  of  COVE  since  it  will  allow  the  ITS  to  train  in  a  manner  similar  to 
fleet  experts.    Finally,  the  creation  of  a  ship-handling  database  for  various  fleet  expert 
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ship-handlers  will  provide  a  solid  footing  for  the  overall  future  development  and 
performance  of  COVE,  increasing  its  quality  as  a  training  tool  and  its  utility  to  the  fleet. 


B.         THESIS  QUESTIONS 

The  following  questions  were  addressed  in  this  thesis: 


•     Is   there   a   relationship   between   one's   personality   and   one's 
expertise? 


Analysis  of  the  NEO-FFI  personality  inventory  administered  to  the 
participant  expert  ship-handlers  indicates  that  there  is  a  relationship 
between  personality  and  expertise.  In  general,  inventoried  experts 
were  less  neurotic  and  more  extroverted  and  conscientious  than  the 
average  adult.  Inventoried  experts  possess  levels  of  Openness  and 
Agreeableness  that  do  not  deviate  from  expected  norms  for  the  average 
adult. 

•     If  such  a  relationship  exists,  can  it  be  quantified? 

The  model  of  an  expert's  personality  should  possess  the  same  levels  of 
Openness  and  Agreeableness  observed  in  the  average  adult,  centering 
primarily  around  average  with  an  approximately  normal  distribution. 
When  modeling  an  expert's  personality,  Neuroticism  should  range 
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from  between  very  low  to  average  with  only  a  small  percentage  of 
experts  possessing  higher  than  average  Neuroticism.  In  contrast,  the 
expert  personality  model  should  possess  levels  of  Extraversion 
primarily  ranging  between  high  and  very  high  with  a  lower  percentage 
of  average  extroverts.  Finally,  the  expert  personality  model  should 
possess  a  primarily  average  or  high  level  of  conscientiousness. 


•     Does    it    extend    beyond    individual    expertise    to    the    expert's 
evaluation  of  others'  performance? 


Analysis  of  the  ship  handling  evaluation  portion  of  the  expert  survey 
indicates  that  there  is  a  relationship  between  personality  and  the 
evaluation  of  performance.  Evaluation  of  the  novice  is  based  upon  a 
combination  of  factors  including  proficiency  of  Interaction, 
Communication,  processing  of  Analytical  and  Sensory  Input,  and  level 
of  adherence  to  the  evaluator's  stylistic  tendencies  as  displayed  by  the 
novice.  The  combination  of  these  factors  is  unique  for  different 
experts  and  is  dependent  upon  the  expert's  personality  profile. 

•     What  is  the  range  of  characteristics  of  different  ship  driving  styles? 

The  data  collected  during  the  ship-handling  survey  spans  a  wide  range 
of  ship-handling  variability  and  will  be  collated  and  sent  to  NAWC- 
TSD  COVE  programmers  for  analysis. 
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The  overall  data  gathered  for  this  experiment  serves  as  a  basis  for  the 
programming  of  an  ITS  for  COVE.  With  this  research,  a  database  now  exists  where  a 
programmer  can  create  a  model  of  ship-handling  expertise  that  is  highly  variable 
depending  upon  user  input.  If  done  correctly,  the  user  could  configure  a  personality 
profile  that  results  in  a  VCO  that  interacts,  evaluations,  and  provides  feedback  to  the 
trainee  in  a  way  that  is  completely  unique  to  the  personality  configuration. 

The  relationship  between  personality,  expertise,  and  evaluation  proposed  in  this 
thesis  will  allow  the  ITS  to  perform  in  a  more  human-like  manner,  increasing  the  illusion 
of  immersion  for  the  trainee,  and  provide  the  trainee  with  accurate  performance 
evaluation  criteria,  increasing  the  positive  training  transfer  benefits.  In  addition  to 
information  about  expertise  and  evaluation,  specific  information  was  gathered  regarding 
preferences  for  acceptable  UNREP  performance.  This  information  will  aid  programmers 
in  determining  acceptable  limits  and  bounds  for  trainees  to  perform  within. 

An  important  point  is  that  this  research  is  entirely  limited  to  COVE  and  UNREP 
evolutions.  While  this  research  utilized  surface  ship-handling  and  COVE  as  a  vehicle, 
the  real  road  is  how  experts  subjectively  evaluate  highly  aggregate  tasks  such  as  UNREP. 
The  theories  and  model  of  expertise  provided  by  this  research  can  potentially  span  across 
any  domain  of  expertise,  as  long  as  the  expertise  is  not  composed  of  black  and  white 
levels  of  performance  and  evaluation. 
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C.         RECOMMENDATIONS  FOR  FUTURE  WORK 

Tenney's  initial  research,  combined  with  the  efforts  of  this  research,  is  a  start  for 
improving  the  man-machine  interface  of  tomorrow's  computer  based  trainers. 
Continuance  of  this  research  could  vector  into  numerous  directions.  Some  possibilities  of 
future  work  include: 


•  Further  investigation  between  Tenney's  VCO  profiles  and  their 
relationship  to  an  expert's  personality. 

•  A  further  analysis  of  the  data  obtained  from  this  experiment 
utilizing  expert  ship  class  as  a  variable 

•  Examining  the  dynamic  between  expert  and  novice  accounting  for 
the  personality  of  the  expert  as  well  as  the  personality  of  the 
novice. 

•  A  linear,  or  non-linear,  regression  model  of  evaluation  with 
coefficients  based  upon  the  correlations  proposed  in  this  research. 

•  An  agent  based  autonomous  agent,  such  as  STEVE,  that  possesses 
a  configurable  personality  within  the  bounds  of  expert  personality 
as  defined  by  this  research. 


Understanding  the  intricacies  of  human  behavior  and  its  relationship  to  highly  aggregate 
tasks  that  are  subjectively  evaluated  is  a  mammoth  effort.  Hopefully,  this  is  only  the 
beginning  of  a  series  that  attempts  to  quantify  how  humans  act  and  react.  Performance  of 
this  type  of  research  is  a  critical  step  in  increasing  the  utility  of  computers  by  making  a 
computer  more  like  a  human  instead  of  a  human  more  like  a  computer. 
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APPENDIX  A.  COMMANDER,  SURFACE  NAVAL  FORCES  ATLANTIC 

ENDORSEMENT 
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COMMANDER  NAVAL  SURFACE  ^CRCE 
U.S.  ATLANTIC  FLEET 

8  May  2000 

Dear  Commanding  Officers. 

The  Naval  Postgraduate  School  (NPS)  maintains  a  proud 
tradition  of  serving  as  a  "Tech  Bridge",  accommodating  advanced 
research  in  direct  support  of  Navy  goals  and  future  development 
The  Conning  Officer  Virtual  Environment  (COVE)  program  is  a 
prime  example  of  how  NPS  research  strives  to  take  Technical  back 
to  Tactical.  Cove  is  a  surface  ship-handling  trainer  that  utilizes 
cutting  edge  computer  graphics  and  physically  based  modeling  to 
simulate  the  ship  driving  experience.  COVE  is  scheduled  for 
imminent  release  to  SWOS  and  the  fleet.  C  A  <ju*»-~  nS^**) 

The  enclosed  survey  is  part  of  an  NPS  thesis  that  directly 
supports  COVE.  The  survey  seeks  to  increase  the  fidelity  and 

realism  of  interactions  between  the  COVE  trainer  and  the  student. 
By  completing  this  survey,  your  time  will  improve  the  quality  of 
this  21st  centurv  readiness  tool. 

\:*i«.  k*.  'kx  3*JL>  A^      Sincerely. 


^0ff  *•  HENRY  C.  GIFFIN  ill 

CIO  a.   v>U  ^W      Vice  Admiral.  U.S.  Navy 
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APPENDIX  B.  COMMANDER,  SURFACE  NAVAL  FORCES  PACIFIC 


ENDORSEMENT 


COMMANDER  NAVAL  SLRFACE  FORCE 
U.S.  PACIFIC  FLEET 


I2\lav2000 


Dear  Commanding  Officer. 


Naval  Postgraduate  School  (NPS)  maintains  a  proud  tradition  of 
serving  as  a  "Tech  Bridge."  accommodating  advanced  research  in  direct 
support  of  Navy  goals  and  future  development.    The  Conning  Officer 
Virtual  Environment  (COVE)  program  is  a  prime  example  of  how  NPS 
research  strives  to  take  technical  back,  to  tactical.  COVE  is  a  surface  ship- 
handling  trainer  that  utilizes  cutting  edge  computer  graphics  and 
physically  based  modeling  to  simulate  the  ship  driving  experience^.  CO VF 
is  scheduled  for  imminent  release  to  SWOS  and  the  fleet.  O&ffjtt. 


ne.t»~.fx-*~ 


h 


The  enclosed  survcj  is  pan  of  a  NPS  thesis  that  directly  supports 
COVE.  The  survey  seeks  to  increase  the  fidelity  and  realism  of 
interactions  between  the  COVE  trainer  and  the  student.  By  completing 
this  survey,  your  time  will  improve  the  qualin  of  this  21s  century- 
readiness  preparation  tool. 


Sincerel 


/ 

MOORE.  JR.       J) 
Vice  Admiral.  U.S.  Navy 
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APPENDIX  C.  EXAMPLE  ADDENDUM  TO  INSTRUCTIONS 


is  Bu/.wk 
(MM  •  SI    N  •       SS  BONHOMV      UCIIAR 

•   v-   x  ■  EXPER1  S 


:  able  or.  the  Internet.         are  cot         s  cipat;  in  the  web  based  ve 

by  typing  ddressimothc  Addrc  'net  Explorer  i  e  Location  Bar 

•lucapeO        ■  ■ 

■  ■ 
the  -U  t     >S  .  ■      ■-  Moves  /ceae  sensitive/. 

choose  to  participate     the  web  based  vc  re  s<  the  survey  packet 

RegardieMOl  (lease  complete  the  survey  no  U  .  Ju'y  1.2000 


rmvcv  of  mow  fiwwt 

smw  «-\^^  era  eukuure 

noviqgs 


% 


--.--- 


.  • £ 


.  \.:::  pie  '  ii  B  Scrccr:  js  v  iewed  by 
Interne;  Explorer 


JCRVfcV  c*  NOW  sxj»ew- 
NOV1CSS 


Figure  2 

Example  Title  Scree::  as  viewed  by 

Netscape  Navigator 


Very  Respectfully  Submitted, 
LT.  Chris  Buziak 
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APPENDIX  D.  EXPERT  SHIP-HANDLING  EVALUATION  SURVEY 


NAVAL  POSTGRADUATE  SCHOOL 

SURVEY  OF  HOW  EXPERT  SHIP  HANDLERS 
EVALUATE  NOVICES 


'*STGBAbU4>. 


"*  _n*^?"    -„  ~£— 


j._^9*^:        -£Km*^ 


?£*% 
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NAVAL  POSTGRADUATE  SCHOOL 

SURVEY  OF  HOW  EXPERT 

SHIP  HANDLERS 

EVALUATE  NOVICES 


INTRODUCTION 

Why  Should  I  Participate? 

Based  c pon  your  experience  -us.,  current 
position,  you  were  identified  ;in  an  expert  ship 
handler.  Your  participation  with  this  survey  tvil] 
easate  that  certain  aspects  of  simulator  training 

arc  realistic  and  pertinent  and  that  simulator - 
trasmng  systems  ate  capable  of  instructing 
trainees  m  a  similar  fashion  to  how  you  instruct 
vour  Junior  Officers' 


pro>  .dzd  this  use  :^  compatible  with  the  purpose 
for  which  the  information  was  collected 

By  participating  in  this  survey,  you 
acknowledge  that  yon  understand  that  your 
privacy  is  mair.taincx.  as  stated  .ib-^vc 
['■urthcrmore.  you  arc  acknowledging  that  your 
participation  is  completely  voluntary. 


INSTRUCTIONS 


How  To  Complete  This  Survey 


COVE  Simulator 

This  survey  supports  development  of  the 
Cocoing  Officer  Virtual  Environment  .COVE). 
sh:p-dr;-.  inc  simulator  scheduled  tor  imminent 
release  to  both  SWOS  and  the  fleet.  COVE  is  a 
fully  funded  training  protect  being  developed  by 
NTS.  Naval  Air  Warfare  Center  Truintng 
Systems  Division  NAWCTSD).  and  the  Office 
of  Naval  Research  <ONR) 

COVE  allows  junior  ship  drivers  the 
Opportunity  to  practice  dangcrnus  ship  handling 
evolutions  cm  a  shore-based  facility  with  no 
danger  to  materia!  or  personnel. 


Privacy  information 

Your  anonytnuy  is  mainiairKrd  .it  all  times 
throughout  this  survey,  and  your  privacy  is 
sxesruarded  under  OPN'W  instruction 
3900.. '<>B  There  is  no  record  linking  your 
participation  with  this  survey  A  record  of  the 
information  contained  :r.  :.1e  experiment 
described  herein  will  be  retained  permanently  at 
the  Nava!  Postgraduate  School  or  by  higher 
authority.  Answers      provided     by      your 

participation  will  be  used  only  for  statistical 
analysis  by  the  Departments  ot  the  Navy  and 
Defense,  and  other  I  S    Government  agencies. 


We  knov.  that  your  time  is  valuable,  this 
survey  should  take  approximately  30  minutes  to 

compie'e    The  survey  consists  of  '.tree  ports 

i  i     Professional  demographic  sccaon 

2>     Personality  inventory 

3  >     L  NREP  e\a-uat:cn  section. 

Completion  of  this  survey  will  require  you 
to  either  til!  m  blank  response  fields  or  complete 
a  multiple  choice  by  circling  your  answer. 
Please  answer  every  question  in  this  survey  Co 
the  best  of  your  ability.  If  there  are  any 
problems  with  this  survey,  please  contact: 

I.T.  Chris  Buziak.  CSN 

O  sJc  32.  Naval  Postgraduate  School 

833  Oyer  Road.  Km  404 

Monterey.  CA  93943-5000 

(S3!  >  656-4679 

DSN  87S-4S79 

I  pen  completion,  please  return  the  survey 
in  the  enclosed  envelope  to  the  above  address. 

Thank  you  tor  your  time  sad  participation. 


Email  any  questions  to  L !   Chris  Buziak  a: 
bu/.iak<=  cs.nps.  navy.mil 
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PROFESSIONAL  DEMOGRAPHICS 


Rank: 


Primary  Warfare  Specialty: 

tclrctf  one* 


Surface 

Aviation 

Submarine 

Other 


Age: 


Sex: 

inrclr  nnt) 


Male 
Female 


Total  Years  of  Officer  Service: 


Number  of  years  in  a  sea  billet: 

(Total) 


Number  of  Commanding  Officers  you 
have  served  under  while  at  sea: 


Current  Sea  Billet: 
Of  a  shore,  fiU  in  SHORE) 


Current  Class  of  Ship: 

/Ilea  short,  fill  in  SHORE » 


Time  since  last  at  sea: 

t  Month]  i 


.Approximate  Number 

of  L'NREP  Approaches 

(Total  during  your  career): 

(cirrlr  rmr) 


0-50 
51-100 

101  -  200 

2O0-?99 

Mote  Than  300 
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PERSONALITY  INVENTORY 

Instructions 

This  section  of  the  survey  consists  of  a  short.  60  qoesnon,  personality  inventory.  This  personality  inventory  is  a 
standardized  professional  psychological  tool  that  NPS  is  utilizing  to  Icam  more  about  ihe  experts  residing  in  the 
fleet.  NPS  believes  that  accurate  modeling  of  expert  leadership  will  enhance  the  fidelity  of  the  COVJi  trainer 
The  inventory  takes  approximately  10  minutes  to  complete  Please  answer  all  of  the  following  60  questions  as 
accurately  and  truthfully  as  possible  selecting  an  answer  that  best  represents  your  opinion  Try  not  to  read  too 
much  into  any  question    Your  first  impression  or  thought  after  reading  the  question  should  be  your  answer.  Do 
not  go  back  and  change  a  previous  answer.  Possible  answers  range  from  strongly  disagree  in  the  leftmost 
column  to  strongly  agree  in  the  rightmost  column 

{SD)  Strongly  Disagree  or  uk  .statement  is  definitely  false 

(D)  Disagree  if  the  statement  a  mostly  false 

(N)  Neutral  li  you  cannot  decide,  or  the  statement  is  equally  uu.:  and  false 

(A)  Agree  cw  (he  statement  is  mostly  true 

(S  A)  Strongly  Agree  or  the  satc-rocot  is  ucJuntcly  true 

Choose  only  one  answer  per  question.  There  is  no  time  limit. 
Inventory 


Question 

SO      D       N 

A      SA 

1    I  am  001  a  wumcr 

2.  I  like  to  have  a  lot  of  people  around  me 

3.  1  donl  like  lo  waste  time  daydrearrang- 

4.  I  try  to  be  courteous  to  everyone  I  meet 

>.  I  keep  my  belongings  clem  and  ocac 

6    1  often  fed  inferior  to  others. 

7.  lUtugh  easily 

8.  Once  1  find  tbe  right  way  to  do  something.  I  stick  to  H 

9.  1  uftcn  get  into  arguments  with  my  tarnily  and  co-wurfcers. 

10   I'm  pretty  good  about  pacing  my  sell  «>  as  to  get  dung?,  done  on  umc. 

1 1    When  I'm  under  a  great  deal  ol  stress,  sometimes  I  feel  bix  I'm  going  to  pieces. 

12,  Idon\coa«io^nn^eUespcciany"ligbi-hcarietr 

I?.  Urn  intrigued  by  the  patterns  I  tind  m  art  and  narurc. 

J 

14    Some  people  think  I'm  selfish  and  egotistical. 

15.  1  am  no*  2  very  methodical  person. 

16    I  rarely  foci  alone  or  blue 

[7.  I  really  enjoy  talking  to  people. 
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(SO)  Strongly  Dtsjorwor  the  siaicmcni  is  ctefrrutely  false 

(D)  Disagree  if  ibe  uaeaciu  is  mostly  false 

(N)  Neutral  if  you  cumoi  decide,  ur  ibe  Hnnmeal  Is  equally  true  and  falve 

(A)  Xgrwnr  che  statement  Is  mostly  true 

(SA)  Strongly  Agra*  a  (be  statcmciii  adefiuiidy  true 


Question 

SD 

D 

N 

A 

SA 

IS   1  bdicvc  icunijt  atudenu  near  controversial  speakers  can  only  coo/uae  and  malead  them. 

IS.  1  >>ouldraU>a<.xiooeriiK*iiJiuLrr:»dijn«xDpc<r  with  them 

20   [  try  lo  perform  all  the  task;  aligned  in  rac  cnnviciiutuily 

21    I  often  feel  tense  and  jram. 

22.  1  lute  to  be  where  etc  ictiwi  is. 

23.  Poetry  bis  link  or  no  effect  on  me 

24.  I  tend  lo  be  cynical  and  -Jtcptical  of  others'  intentions. 

Z\  T  ha\e  a  clear  «  of  poll  and  wort  toward  (here  in  an  orderly  Xacbion. 

26  Sometimes  1  fed  completely  »t«hlcw 

27.  I  unratly  prefer  to  do  things  alone. 

28.  I  often  cry  new  and  foraro  foods. 

— 

29.  1  hclt  c  ve  thai  ox  At  people  win  rake  advantage  of  yon  if  yon  lei  Ibem 

30.  (wastes  lot  of  nine  before  settling  oowuloweri. 

31    I  tardy  Icel  fearful  <w  anxious. 

32.  [  often  feel  as  if  I'm  burning  with  energy. 

33.  I  seldom  nonce  the  roods  or  fcchiuts  that  rifferenl  environment*  produce. 

34  Mow  people  1  know  lite  me. 

35    I  work  hard  lo  accotupusli  my  goals. 

36.  I  uften  get  angry  at  Lot  way  people  seal  me. 

37.  1  am  a  cbeerrul.  high-spirited  person. 

38.  1  believe  wc  should  look  to  our  religious  authorities  for  decisions  on  moral  issues. 

39.  Soane  people  Until,  of  me  w  culu  and  ealculaung. 

40.  When  1  malce  a  cotTunlmrm  I  can  always  Be  counted  on  to  follow  through 

41.   loonncn.  when  things  go  wrong,  1  g«  dis»xrate<1  and  fed  hke  giving  op. 

42.  1  are  out  i  cheerful  optimist 

43    Sometimes  wben  1  am  reading  poetry  or  looking  ma  work  of  an,  I  fed  a  chill  or  wave  of 
CaeilcmiTtL 
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(SO)  Strongly  Disagree  i»  ihc  snwmaH  is  definitely  false 

(0)  Disagree  if  (he  waiemnn  is  mostly  lib* 

(N)  Neutral  tf  yon  cannot  decidv,  of  the  statement  a  equally  true  and  false 

(A)  Agree  or  the  statement  is  mostly  true 

{SA>  Strongly  Agree  or  the  statement  is  definitely  tnie 


Question                                                 SD 

D       N       A 

SA 

44    Km  bard-- headed  and  tough- nnnded  to  ra>  altitudes. 

45    Sometimes  I'm  not  as  dependable  or  reliable  as  1  should  be 

46    I  am  seldom  wd  or  depressed. 

47.  Mv  Itfe  u  fast-paced 

1 

1 

4S    I  have  tittle  iiucdi  in  speculating  on  the  nature  of  the  universe  ex  die  human  condition      j 

-i  ■    I  generally  try  to  be  thoughtful  and  considerate 

50    I  am  a  productive  person  who  always  get*  the  job  done 

i         j 

51    1  otien  feci  helpless  and  want  someone  else  to  solve  my  problems. 

i 

52    1  am  a  very  active  person 

I 

5?    i  tei\T  a  lot  (x  intellectual  curiosity 

1 

M    If  1  don*  Like  people.  I  let  then  know  «. 

i 

55,  I  never  seem  to  be  iMe  to  get  organized 

56.  At  times  [have  been  so  ashamed  I  just  want  to  bide 

j 

57.  I  would  rather  go  my  own  way  then  be  a  leader  of  others 

58    I  often  enjoy  playing  with  theories  or  abstract  ideas. 

5°    If  necessary. '  am  willing  to  manipulate  peorttc  to  get  what  1  want. 

60    1  strive  for  excellence  in  everything  I  da 
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UNREP  EVALUATION  SURVEY 
Introduction 

This  portion  of  the  survey  deals  with  how  you  would  evaluate  a  junior  ship  handler   For  the  purpose  of  this 
survey,  imagine  that  you  have  just  supervised  a  young,  less  experienced  junior  officer  who  has  completed  the 
approach  on  an  AE    Following  completion  of  the  evolution,  you  are  tasked  with  evaluating  the  JOs 
performance  and  providing  the  JO  with  feedback  about  the  execution  of  the  approach.  This  portion  of  the 
survey  is  intended  to  determine  the  specific  criteria  you  use  to  evaluate  a  conning  officer  s  UNREP  approach 
performance. 

This  inventory  consist  of  3  pans.  While  the  entire  inventory  requires  only  about  15  minutes,  there  is  no  time 
limit.  Please  answer  each  of  the  following  questions  considering  ycur  requirements  for  a  successful  approach 


Parti 

Part  I  requires  you  to  evaluate  each  of  the  following  3  UNREP  approach  templates.  Evaluation  criteria  for 
each  of  the  three  UNREP  styles  are: 

Preferred  -  This  method  roost  closely  matches  how  you  expect  your  Conning  Officer  to  make  the  L'NREP  approach. 
Acceptable  -  Tlus  method  nay  not  be  exactly  bow  you  expect  the  approach  to  occur,  but  you  nave  no  objections  u>  this  type  of 

execution 
Unacceptable  -  You  will  not  allow  a  Coming  Officer  to  execute  (he  UN'REP  in  its-,  fashion. 


Lp%erA  Far  Diagram* 
RECOMMEND  ICOURSn    SPCED1 

Course 

Speed 

1   I.- Left 

1-  Increase 

i  R    Righ! 

D    Dvcrcue 

I   N  -  No  Chance 

N  -  No  Charree 
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120  feet 

! 

[^^^    CLOtE     twrmwunn  in 

^^■i   BPBO    tfrnwumTD  LIU 
1000  feet 

^■■"   as  i/on    mnm  a.  hi 

ISOOfeet          _/\ 

2000  feet              LJ 

Pi 

(circlr  an) 
efeired                                                     Acceptable 

Unacceptab 

lc 
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1 20  feet 


LEccHunrop-D] 

AXCBCtlDE 


Ofeet 


1000  feet 


1800  feet 
2000  feet 


BUAW.W.Y    XZCOMHEHDIHJ 
PMEIZM&.W.Y    BBO— MPPTW 


rLOtt    larormmnjaji 


rnrniy    izcatmurD  a.m 


rariKHT    tfncwimg)  g.  m 


Prefened 


Acceptable 


Unacceptable 
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120  feet 


Ofeet 


bixakkwxy    ixccaaaxDMV, 
Bt£MUC>w.Y    to:  ceaanD  mtn 

XI£JHOCDE     MP  WM»Hnmin 


CXOfr      MCnMMTVDg.il 


MirmtT     MCQUUHn)  IH  H) 


100  Ofeet 


nWIAHI     XKGWEHD1HH1 


1800 
2000  feet 


Preferred 


icisrir  onsi 

Acceptable 


Unacceptable 
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Part  II 

Part  II  covers  specific  criteria  that  may  be  used  10  evaluate  a  Conning  Officer   Possible  answers  for  each  of  the 
following  questions  in  Part  II  are: 

Not  Applicable  (N/A)  -There  is  do  need  10  perform  the  acnon. 

Applicable  (A)  -  Tins  in  a  relatively  minor  action  with  Urge  room  fur  variation  ol  esecuuon 

Somewhat  Important  (SI)  -  An  «n»  that  mist  be  perfortned  to  have  a  successful  L'Nkl:  I'  approach,  txil 

with  some  room  for  variation  ol  cxcciraan. 

Important  (I)  -  This  action  inust  he  performed  well  to  have  a  successful  approach  with  little  variation  of 

execution 

Critical  (C)  •  It  is  imposstWe  to  successfully  complete  an  L'NRiiP  approach  without  performing  this  action 

flawlessly. 

Please  answer  each  of  the  following  questions  by  providuig  one  of  the  above  ratings.  If  your  ship  does  not  have 
a  required  piece  of  equipment  to  perform  an  action,  answer  as  if  the  required  gear  was  installed  on  your  ship. 


Conning  Officer 
Preparations/Qualifications 

while  you  do  not  necessarily  expect  the  Conning 

Offlcr  to  perform  these  actions.  ihe\  are  knowledge 

requirements  for  a  good  Conning  Officer 

N/A                 A                  SI 

i 

o 

1.  Possesses  an  m-tlepUi  know ledge  of  own  ship's 
numeric  rates  of  accelcrauoti/uccclcrauon  and  other 
handling  characteristics. 

~  ~r~  i "  ~ 

2.  Possesses  an  lu-dcpth  knowledge  of  handutuj 
characteristics  of  auxiliary  ship  le.q.  acceleration  and 
turn  nsiesj. 

3.  Possesses  an  understanding  bow*  Own  Ship's 
vtbraoons  correspond  to  speed  and  rudder  commands 

4    Possesses  an  ability  to  sense  environmenul  factors 

5.  Possesses  an  understanding  of  wind  force  effects 
on  ship's  freeboard 

6   Possesses  understanding  of  depth  of  ocean  effects 

7.  Possesses  understanding  of  topography  of  ocean 
floor  effects 

1 

•-.   Possesses  understanding  of  sea  sale  effects. 

i        i 

1 

0   Possesses  understanding  of  direction  of  seas 
effects. 

i 

10.  Possesses  understanding  of  Bernoulli  forces 

1 

11.  Possesses  understanding  of  M/lntegraicd  ship's 
circuit. 

1 2    Possesses  understanding  of  radio  hcadsct/waj  bc- 
ullac  for  internal  comrmmicalions. 

1 3.  Possesses  understanding  of  Bridge  to  Bridge 
handsel 

j 

10 
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Not  Applicable  (N/A)  -There  is  no  need  \o  perform  ihu  action. 

Applicable  (A)  -  This  is  a  relatively  ounce  action  with  large  room  for  variation  of  execution. 

Somewhat  Important  (Sf)  -  An  action  Qui  mu*i  be  pert'nrned  lr>  have  a  successful  UNRFP  approach,  but 

with  some  room  for  variation  of  csocuuua 

Important  (I)  -  This  action  must  be  pcrfonncd  well  to  have  a  successful  approach  with  link  vuhaUoo  of 

cscctjuotl 

Critical  (C)  -  It  ti  impossible  to  successfully  complete  an  UNRfcP  approach  wtlhout  perforrrnm;  this  action 

flawless)  v. 


Conning  Officer 
Preparations/Qualifications 

Whtle  you  do  not  necessarily  expect  the  Conning 

Officer  ;o  perform  tte.se  actions,  they  are  knowledge 

requirements  for  a  good  Conning  Officer 

N/A 

1             j 

A                  SI                   1 

c 

1 4    Poyscsses  understanding  of  UHF 

1 5    Possesses  understanding  of  crllular 
comnmicttiaiL 

i 

16.  iVisexscs  understanding  of  Lights  (for 

communication). 

' 

1 7    Possesses  understanding  of  semaphores. 

I 

1 8    Possesses  understanding  of  formal  decision  aids 
'information  boards.  cherJctixt.r.  flcr*.  charts  etc  A 

i 

19    Possesses  understanding  of  NTVP  procedure 

i 

20.  Possesses  understanding  of  SOPs. 

2 1    Displayed  proper  use  of  MOSQARD 

II 
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Not  Applicable  (NM)  -There  b  do  need  10  perform  irm  acuon. 

Applicable  (A) .  This  is  j  relative)  v  minor  action  with  Urge  room  for  variation  of  execution 

Somewhat  Important  (SI)  -  An  action  Oat  must  be  performed  lo  have  a  successful  UNKfcP  approach,  but 

with  .Home  room  for  variation  ot  ex  centum. 

Important  (I)  -  This  action  must  be  performed  well  to  have  a  successful  approach  with  utile  vanahon  of 

execution 

Critical  (C)  -  It  is  impossible  to  successfully  complete  an  UNRfcP  approach  without  performing  this  action 

flawlessly 


Execution 

Their  or'  actions  you  expert  a  good  Conning  Officer 
lu  perform  well  Ounnit  lh<  \}»REP  approach 

N/A 

A 

SI 

1               c 

I,  Maintained  cioxc  verhal  communication  between 
Com  and  CO/XO  durmg  approach 

2    Maintained  regular  communications  between  own 
>hip  and  Auxiliary  durui£  approach. 

?.  Displayed  propa  use  of  face  to  face 
communications. 

i 

4.  Displayed  proper  use  of  J  A/lmcgraicd  snip's 
or  curt. 

5.  Displayed  proper  use  of  radio  headxet/wallae- 
LalxjL-  for  uitemal  eunuxiumcauons. 

! 

6   Displayed  proper  use  ot  Bridge  to  Bridge  handset. 

7    Displayed  proper  use  at  UHK 

: 

S   Displayed  proper  use  of  cellular  communication. 

9    Displayed  proper  use  of  lights  (for 

communicatiani. 

i 

1 0   Displayed  proper  use  of  semaphores. 

11.  Displayed  proper  use  of  formal  decision  aids 
tinformaaon  hoards,  checklist*,  flrrw  charts  etc-.,  i. 

i 

11  Displayed  proper  formal  use  of  NWP  procedure 

13.  Displayed  proper  use  of  SOPs. 

!          ; 

14.  Displayed  proper  use  of  rules  of  thumb  (non- 
numenc.  nan-calculation  basedi 

!                      i 

1 5.  Displayed  proper  use  of  radian  rule  (and  any 
other  calculation  based  rule  of  thumb). 

1 6.  Displayed  proper  use  of  an  internal  checklist  or 
other  svstcmatic  routine  fur  compleuon  of  rmlesiones 

■                     i 

17.  Displayed  proper  use  of  a  scan  routine  for  input 

i                     1 

18   Displayed  proper  visual  observation  of  w-ale  u> 
rudjfe  relahve  rnouon. 

1             : 

19  Displayed  proper  visual  observation  of  relative 
perspective  to  tudRe  relative  motion 

20.  Displayed  proper  visual  observation  of  rate  of 
closure  to  luditc  relative  motion. 

21    Displayed  proper  and  umely  use  oi GPS 
position/speed 

22.  Displayed  proper  use  of  MOBOARD. 

1              ; 

23    Displayed  propci  and  nmelv  use  o!  Sodim-in 
information. 

! 
i 

24  Displayed  propa  and  bmely  use  of  Radar  range 
informauon 

25.  Decayed  proper  visual  estimations  at  range. 

26   Displayed  proper  and  bmely  use  of  laser 
ranxetindcr  information. 

|                j 

12 
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Part  III 

Part  III  consists  of  some  general  questions  about  UNREP  approaches.  Answers  for  Part  III  consist  of  multiple 
choice  and  fill  in  the  blank  that  best  correspond  with  your  expert  style  of  ship  driving. 


What  is  your  minimum  allowable  lateral  separation? 

tinjiel) 

What  is  your  maximum  allowable  lateral  separation'' 

'In  fttt) 

What  is  your  ideal  approach  speed  differential? 

What  is  your  ideal  distance  behind  the 
Auxiliary  prior  to  commencement  of  approach? 

tm  \ardv 

What  is  an  adequate  amount  of  lime  to  perform  an 

approach  for  a  new  Junior  Officer? 

(circle  one) 

Under  5  minutes 
Between  5  and  10  minutes 

Between  10  and  15  minutes 
Between  15  and  20  mlnuias 

Greater  Than  20  minutes 
Time  is  a  luncDon  of  speed  and  distance 

There  is  no  'adequate*  amount  ol  time 

What  is  an  adequate  amount  of  tune  to  perform  an 
approach  for  an  experienced  Senior  Officer? 

(circle  nne> 

Under  5  minutes 
Between  5  and  to  minutes 

Between  10  and  15  minutes 
Between  1 5  and  20  minutes 

Greater  Tnan  20  minutes 
Time  is  a  function  of  speed  and  distance 

There  is  no  "adequate*  amount  ot  time 

How  would  you  characterize  your  management  of  Conning 
Officers  during  an  UNREP? 

(circle  one) 

I  allow  large  variations  In 
execution  from  my  preferences 

1  allow  some  variations  in 
execution  from  my  preferences 

I  allow  no  variations  in 
execution  from  my  preferences 

13 
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NVVP  Is  a  detailed  recipe  for  UNREP 

Wh,ch  statement  best  describes  tbe  fWVP?  i  nwp  ,s  only  a  loose  framework  for  UNREP 

(circle  one)  ! 


A  conning  officer  who  primarily  uses  rules  of  thumb. 
;  ships  instruments  and  other  guidelines  to  make 
decisions'' 

(someone  v/ho  knows  numbers  ant}  loots) 
Which  are  you  most  comfortable  with:  j 

(circle  one)  j  or 

A  conning  officer  who  primarily  uses  hts  instincts  and 
sensory  estimations? 
(someone  who  drives  a  ship  like  a  car  on  me  highway) 


CONCLUSION 


Thank  you  for  taking  your  valuable  time  to 
complete  this  survey.  Tbe  goal  of  this  research  is  to  LT.  Chris  BuziaJk.  USN: 

determine  how  to  tailor  feedback  provided  to  a  Junior  Code  32.  Naval  Postgraduate  School 

Officer  (JO)  during  training  in  a  Virtual  Environment  833  Dyer  Road  Rm  404 

(VE)  ship  handling  training  simulator  such  as  Monterey.  CA  93943-5000 

NAWCTSD  s  Conning  Officer  Virtual  Environment  (83 1 )  656^679       DSN  878-4679 

(COVE).  Tailored  feedback  allows  a  simulator  to 

train  a  JO  in  a  way  that  is  compatible  with  the  JO's  Please  encourage  your  expert  peers  to  participate 

Commanding  Officer  s  ship  handling  style.  If  a  JO  in  this  survey.  Greater  expert  participation  will 

can  receive  feedback  that  is  similar  to  the  feedback  improve  tbe  results  and  create  a  better  product 

he  receives  from  his  shipboard  mentor,  the  JO  will  released  to  the  fleet, 

have  more  effective  simulator  training  time  and  be 
more  productive  while  at  sea, 

lf  you  would  like  to  leave  suggestions  or 
comments,  feel  free  to  enter  them  on  the  back  of  this 
survey: 


For  additional  questions  or  comments  about  this  ~    Email  any  questions  to  LT.  Chris  Buziak  at 

research,  please  contact:  buziak@cs.nps.navy .mil 


14 
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Enter  Your  Comments  Here 


is 
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APPENDIX  E.  UNREP  CHARACTERISTICS  BASED  UPON 


APPROACH  SHIP  CLASS 


SMALLEST  OBSERVED 
MINIMUM  ALLOWABLE 
LATERAL  SEPARATION 

(FEET) 

AVERAGE  MINIMUM 

ALLOWABLE  LATERAL 

SEPARATION 

(FEET) 

LARGEST  OBSERVED 

MINIMUM  ALLOWABLE 

LATERAL  SEPARATION 

(FEET) 

CG 

80 

117 

140 

CV/CVN 

120 

136 

140 

DD/DDG 

80 

122 

150 

FFG 

60 

109 

160 

Transport  Ship 

80 

135 

180 

Table  7.  Minimum  Allowable  Lateral  Separation 

Between  Approach  Ship  and  Replenishment  Ship 

Based  Upon  Approach  Ship  Class 


SMALLEST  OBSERVED 

MAXIMUM  ALLOWABLE 

LATERAL  SEPARATION 

(FEET) 

AVERAGE  MAXIMUM 

ALLOWABLE  LATERAL 

SEPARATION 

(FEET) 

LARGEST  OBSERVED 

MAXIMUM  ALLOWABLE 

LATERAL  SEPARATION 

(FEET) 

CG 

160 

185 

200 

CV/CVN 

180 

200 

220 

DD/DDG 

140 

189 

250 

FFG 

110 

182 

240 

Transport  Ship 

140 

203 

300 

Table  8.  Maximum  Allowable  Lateral  Separation 

Between  Approach  Ship  and  Replenishment  Ship 

Based  Upon  Approach  Ship  Class 
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SMALLEST  OBSERVED 

SPEED  DIFFERENTIAL 

(KNOTS) 

AVERAGE  OBSERVED 

SPEED  DIFFERENTIAL 

(KNOTS) 

LARGEST  OBSERVED 

SPEED  DIFFERENTIAL 

(KNOTS) 

CG 

5 

9 

18 

CV/CVN 

3 

5 

6 

DD/DDG 

5 

7 

20 

FFG 

5 

11 

25 

Transport  Ship 

3 

6 

18 

Table  9.  Allowable  Approach  Speed  Differential 

Between  Approach  Ship  and  Replenishment  Ship 

Based  Upon  Approach  Ship  Class 


SMALLEST  OBSERVED 

STARTING  DISTANCE 

(YARDS) 

AVERAGE  OBSERVED 

STARTING  DISTANCE 

(YARDS) 

LARGEST  OBSERVED 

STARTING  DISTANCE 

(YARDS) 

CG 

400 

671 

1000 

CV/CVN 

1000 

1800 

2000 

DD/DDG 

160 

623 

3000 

FFG 

300 

652 

1500 

Transport  Ship 

300 

748 

1800 

Table  10.  Allowable  Starting  Distance  For  Approach 

Between  Approach  Ship  and  Replenishment  Ship 

Based  Upon  Approach  Ship  Class 
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